Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_single_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_single_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 3,
 'device': device(type='cuda', index=1),
 'dropout': 0.35000000000000003,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_single/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1600,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1600,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_single_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_single/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_single/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.310740
Epoch 1
Validation binary_cross_entropy = 3.521893
Epoch 2
Validation binary_cross_entropy = 88.056107
Epoch 3
Validation binary_cross_entropy = 6.038639
Epoch 4
Loss = 5.9451e+01, PNorm = 82.9669, GNorm = 1456.1935, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 6.916775
Epoch 5
Validation binary_cross_entropy = 67.264633
Epoch 6
Validation binary_cross_entropy = 46.741806
Epoch 7
Validation binary_cross_entropy = 6.415419
Epoch 8
Validation binary_cross_entropy = 17.366190
Epoch 9
Loss = 3.7187e+01, PNorm = 83.2396, GNorm = 1056.6898, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 57.725456
Epoch 10
Validation binary_cross_entropy = 5.906857
Epoch 11
Validation binary_cross_entropy = 8.774070
Epoch 12
Validation binary_cross_entropy = 53.876728
Epoch 13
Validation binary_cross_entropy = 17.880878
Epoch 14
Loss = 2.4048e+01, PNorm = 83.5420, GNorm = 711.2159, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 5.996768
Epoch 15
Validation binary_cross_entropy = 18.766741
Epoch 16
Validation binary_cross_entropy = 46.353348
Epoch 17
Validation binary_cross_entropy = 8.106318
Epoch 18
Validation binary_cross_entropy = 7.464089
Epoch 19
Loss = 9.9602e+00, PNorm = 83.7601, GNorm = 485.2487, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 38.893631
Epoch 20
Validation binary_cross_entropy = 10.409101
Epoch 21
Validation binary_cross_entropy = 8.186019
Epoch 22
Validation binary_cross_entropy = 17.680826
Epoch 23
Validation binary_cross_entropy = 20.016380
Epoch 24
Loss = 1.2351e+01, PNorm = 83.9100, GNorm = 502.6822, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.088798
Epoch 25
Validation binary_cross_entropy = 5.197368
Epoch 26
Validation binary_cross_entropy = 7.460179
Epoch 27
Validation binary_cross_entropy = 6.829323
Epoch 28
Validation binary_cross_entropy = 5.383762
Epoch 29
Loss = 1.3255e+01, PNorm = 84.0252, GNorm = 464.5173, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 5.083975
Epoch 30
Validation binary_cross_entropy = 4.549232
Epoch 31
Validation binary_cross_entropy = 4.986238
Epoch 32
Validation binary_cross_entropy = 12.667368
Epoch 33
Validation binary_cross_entropy = 8.361389
Epoch 34
Loss = 2.0551e+01, PNorm = 84.1307, GNorm = 467.9950, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 4.579204
Epoch 35
Validation binary_cross_entropy = 3.734390
Epoch 36
Validation binary_cross_entropy = 6.290204
Epoch 37
Validation binary_cross_entropy = 13.661296
Epoch 38
Validation binary_cross_entropy = 3.201177
Epoch 39
Loss = 7.4957e+00, PNorm = 84.2440, GNorm = 369.6091, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 3.338012
Epoch 40
Validation binary_cross_entropy = 3.897730
Epoch 41
Validation binary_cross_entropy = 4.772079
Epoch 42
Validation binary_cross_entropy = 3.858742
Epoch 43
Validation binary_cross_entropy = 3.032702
Epoch 44
Loss = 4.3623e+00, PNorm = 84.3443, GNorm = 150.0927, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 3.997864
Epoch 45
Validation binary_cross_entropy = 3.479967
Epoch 46
Validation binary_cross_entropy = 5.665185
Epoch 47
Validation binary_cross_entropy = 5.641126
Epoch 48
Validation binary_cross_entropy = 3.180133
Epoch 49
Loss = 4.3307e+00, PNorm = 84.4193, GNorm = 223.0717, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 2.501311
Epoch 50
Validation binary_cross_entropy = 2.850525
Epoch 51
Validation binary_cross_entropy = 5.172219
Epoch 52
Validation binary_cross_entropy = 4.428320
Epoch 53
Validation binary_cross_entropy = 3.103034
Epoch 54
Loss = 2.1046e+00, PNorm = 84.5049, GNorm = 115.0799, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 1.978142
Epoch 55
Validation binary_cross_entropy = 1.983814
Epoch 56
Validation binary_cross_entropy = 1.645446
Epoch 57
Validation binary_cross_entropy = 2.095611
Epoch 58
Validation binary_cross_entropy = 1.511299
Epoch 59
Loss = 4.2595e+00, PNorm = 84.6132, GNorm = 149.3274, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 1.585415
Epoch 60
Validation binary_cross_entropy = 3.036638
Epoch 61
Validation binary_cross_entropy = 4.736655
Epoch 62
Validation binary_cross_entropy = 3.212837
Epoch 63
Validation binary_cross_entropy = 1.605110
Epoch 64
Loss = 3.6712e+00, PNorm = 84.7392, GNorm = 208.4672, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 1.737249
Epoch 65
Validation binary_cross_entropy = 4.572145
Epoch 66
Validation binary_cross_entropy = 8.119340
Epoch 67
Validation binary_cross_entropy = 4.528557
Epoch 68
Validation binary_cross_entropy = 1.737437
Epoch 69
Loss = 5.8631e+00, PNorm = 84.8415, GNorm = 197.6897, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 1.697865
Epoch 70
Validation binary_cross_entropy = 1.349663
Epoch 71
Validation binary_cross_entropy = 2.508692
Epoch 72
Validation binary_cross_entropy = 3.122672
Epoch 73
Validation binary_cross_entropy = 1.762388
Epoch 74
Loss = 3.3089e+00, PNorm = 84.9651, GNorm = 112.1130, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.907129
Epoch 75
Validation binary_cross_entropy = 0.851794
Epoch 76
Validation binary_cross_entropy = 0.906021
Epoch 77
Validation binary_cross_entropy = 1.631888
Epoch 78
Validation binary_cross_entropy = 1.854268
Epoch 79
Loss = 3.0017e+00, PNorm = 85.0582, GNorm = 79.0086, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.737487
Epoch 80
Validation binary_cross_entropy = 2.418572
Epoch 81
Validation binary_cross_entropy = 2.483471
Epoch 82
Validation binary_cross_entropy = 1.580315
Epoch 83
Validation binary_cross_entropy = 1.430680
Epoch 84
Loss = 2.5535e+00, PNorm = 85.1383, GNorm = 111.3903, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.884930
Epoch 85
Validation binary_cross_entropy = 0.792528
Epoch 86
Validation binary_cross_entropy = 0.941077
Epoch 87
Validation binary_cross_entropy = 0.962820
Epoch 88
Validation binary_cross_entropy = 0.810744
Epoch 89
Loss = 1.7359e+00, PNorm = 85.2613, GNorm = 90.9236, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.880875
Epoch 90
Validation binary_cross_entropy = 0.889923
Epoch 91
Validation binary_cross_entropy = 0.894557
Epoch 92
Validation binary_cross_entropy = 0.774176
Epoch 93
Validation binary_cross_entropy = 0.800616
Epoch 94
Loss = 1.4115e+00, PNorm = 85.3954, GNorm = 90.5120, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.826314
Epoch 95
Validation binary_cross_entropy = 0.796540
Epoch 96
Validation binary_cross_entropy = 0.726452
Epoch 97
Validation binary_cross_entropy = 0.731045
Epoch 98
Validation binary_cross_entropy = 0.776334
Epoch 99
Loss = 1.7820e+00, PNorm = 85.5739, GNorm = 80.9531, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.598189
Epoch 100
Validation binary_cross_entropy = 2.007807
Epoch 101
Validation binary_cross_entropy = 1.520901
Epoch 102
Validation binary_cross_entropy = 0.851154
Epoch 103
Validation binary_cross_entropy = 0.705246
Epoch 104
Loss = 7.5494e-01, PNorm = 85.6840, GNorm = 40.4930, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.697835
Epoch 105
Validation binary_cross_entropy = 0.720792
Epoch 106
Validation binary_cross_entropy = 1.256423
Epoch 107
Validation binary_cross_entropy = 4.351655
Epoch 108
Validation binary_cross_entropy = 5.257760
Epoch 109
Loss = 3.5846e+00, PNorm = 85.7875, GNorm = 146.6149, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.292816
Epoch 110
Validation binary_cross_entropy = 0.574323
Epoch 111
Validation binary_cross_entropy = 0.688227
Epoch 112
Validation binary_cross_entropy = 0.608956
Epoch 113
Validation binary_cross_entropy = 0.605229
Epoch 114
Loss = 1.2048e+00, PNorm = 85.8698, GNorm = 81.1786, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.400304
Epoch 115
Validation binary_cross_entropy = 1.726942
Epoch 116
Validation binary_cross_entropy = 1.654337
Epoch 117
Validation binary_cross_entropy = 0.738934
Epoch 118
Validation binary_cross_entropy = 0.591394
Epoch 119
Loss = 1.5669e+00, PNorm = 85.9274, GNorm = 69.9018, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.571622
Epoch 120
Validation binary_cross_entropy = 0.562686
Epoch 121
Validation binary_cross_entropy = 0.560410
Epoch 122
Validation binary_cross_entropy = 0.563920
Epoch 123
Validation binary_cross_entropy = 0.581209
Epoch 124
Loss = 1.5795e+00, PNorm = 86.0077, GNorm = 88.5127, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.606071
Epoch 125
Validation binary_cross_entropy = 0.697256
Epoch 126
Validation binary_cross_entropy = 0.753514
Epoch 127
Validation binary_cross_entropy = 0.705030
Epoch 128
Validation binary_cross_entropy = 0.584579
Epoch 129
Loss = 7.3886e-01, PNorm = 86.0776, GNorm = 51.2238, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.458857
Epoch 130
Validation binary_cross_entropy = 0.414705
Epoch 131
Validation binary_cross_entropy = 0.411372
Epoch 132
Validation binary_cross_entropy = 0.473146
Epoch 133
Validation binary_cross_entropy = 0.573290
Epoch 134
Loss = 1.4587e+00, PNorm = 86.1754, GNorm = 55.2077, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.646166
Epoch 135
Validation binary_cross_entropy = 0.683100
Epoch 136
Validation binary_cross_entropy = 0.583545
Epoch 137
Validation binary_cross_entropy = 0.430480
Epoch 138
Validation binary_cross_entropy = 0.364105
Epoch 139
Loss = 1.1758e+00, PNorm = 86.2622, GNorm = 53.4145, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.357136
Epoch 140
Validation binary_cross_entropy = 0.380297
Epoch 141
Validation binary_cross_entropy = 0.418307
Epoch 142
Validation binary_cross_entropy = 0.421009
Epoch 143
Validation binary_cross_entropy = 0.407527
Epoch 144
Loss = 7.6436e-01, PNorm = 86.3541, GNorm = 38.7227, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.394218
Epoch 145
Validation binary_cross_entropy = 0.373607
Epoch 146
Validation binary_cross_entropy = 0.393350
Epoch 147
Validation binary_cross_entropy = 0.419429
Epoch 148
Validation binary_cross_entropy = 0.463706
Epoch 149
Loss = 5.1142e-01, PNorm = 86.4521, GNorm = 42.4386, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.482191
Epoch 150
Validation binary_cross_entropy = 0.395411
Epoch 151
Validation binary_cross_entropy = 0.407198
Epoch 152
Validation binary_cross_entropy = 0.474484
Epoch 153
Validation binary_cross_entropy = 0.464411
Epoch 154
Loss = 1.0168e+00, PNorm = 86.5822, GNorm = 71.2948, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.438188
Epoch 155
Validation binary_cross_entropy = 0.572164
Epoch 156
Validation binary_cross_entropy = 0.639282
Epoch 157
Validation binary_cross_entropy = 0.554058
Epoch 158
Validation binary_cross_entropy = 0.451734
Epoch 159
Loss = 1.0134e+00, PNorm = 86.7351, GNorm = 68.1986, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.398843
Epoch 160
Validation binary_cross_entropy = 0.398198
Epoch 161
Validation binary_cross_entropy = 0.427625
Epoch 162
Validation binary_cross_entropy = 0.448506
Epoch 163
Validation binary_cross_entropy = 0.402219
Epoch 164
Loss = 6.1396e-01, PNorm = 86.9068, GNorm = 41.2714, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.397724
Epoch 165
Validation binary_cross_entropy = 0.368424
Epoch 166
Validation binary_cross_entropy = 0.361422
Epoch 167
Validation binary_cross_entropy = 0.400009
Epoch 168
Validation binary_cross_entropy = 0.546234
Epoch 169
Loss = 4.1971e-01, PNorm = 87.1041, GNorm = 22.7180, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.674332
Epoch 170
Validation binary_cross_entropy = 0.549302
Epoch 171
Validation binary_cross_entropy = 0.430683
Epoch 172
Validation binary_cross_entropy = 0.377988
Epoch 173
Validation binary_cross_entropy = 0.355553
Epoch 174
Loss = 9.4500e-01, PNorm = 87.2416, GNorm = 37.4018, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.339553
Epoch 175
Validation binary_cross_entropy = 0.426220
Epoch 176
Validation binary_cross_entropy = 0.696638
Epoch 177
Validation binary_cross_entropy = 0.896962
Epoch 178
Validation binary_cross_entropy = 0.764775
Epoch 179
Loss = 7.5469e-01, PNorm = 87.3843, GNorm = 55.5049, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.597408
Epoch 180
Validation binary_cross_entropy = 0.501089
Epoch 181
Validation binary_cross_entropy = 0.443838
Epoch 182
Validation binary_cross_entropy = 0.391893
Epoch 183
Validation binary_cross_entropy = 0.370344
Epoch 184
Loss = 5.9884e-01, PNorm = 87.5310, GNorm = 52.9757, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.376923
Epoch 185
Validation binary_cross_entropy = 0.379869
Epoch 186
Validation binary_cross_entropy = 0.385787
Epoch 187
Validation binary_cross_entropy = 0.406030
Epoch 188
Validation binary_cross_entropy = 0.425353
Epoch 189
Loss = 8.5552e-01, PNorm = 87.6932, GNorm = 62.8939, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.387322
Epoch 190
Validation binary_cross_entropy = 0.348331
Epoch 191
Validation binary_cross_entropy = 0.349471
Epoch 192
Validation binary_cross_entropy = 0.360938
Epoch 193
Validation binary_cross_entropy = 0.360526
Epoch 194
Loss = 5.4387e-01, PNorm = 87.8459, GNorm = 56.6629, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.366386
Epoch 195
Validation binary_cross_entropy = 0.382238
Epoch 196
Validation binary_cross_entropy = 0.453002
Epoch 197
Validation binary_cross_entropy = 0.452496
Epoch 198
Validation binary_cross_entropy = 0.447166
Epoch 199
Loss = 5.6188e-01, PNorm = 87.9885, GNorm = 35.1339, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.424116
Epoch 200
Validation binary_cross_entropy = 0.428978
Epoch 201
Validation binary_cross_entropy = 0.446620
Epoch 202
Validation binary_cross_entropy = 0.450604
Epoch 203
Validation binary_cross_entropy = 0.375989
Epoch 204
Loss = 4.2416e-01, PNorm = 88.0887, GNorm = 33.4753, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.329553
Epoch 205
Validation binary_cross_entropy = 0.332699
Epoch 206
Validation binary_cross_entropy = 0.329163
Epoch 207
Validation binary_cross_entropy = 0.336333
Epoch 208
Validation binary_cross_entropy = 0.384059
Epoch 209
Loss = 3.3541e-01, PNorm = 88.2036, GNorm = 12.6424, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.453047
Epoch 210
Validation binary_cross_entropy = 0.456948
Epoch 211
Validation binary_cross_entropy = 0.392664
Epoch 212
Validation binary_cross_entropy = 0.379922
Epoch 213
Validation binary_cross_entropy = 0.362881
Epoch 214
Loss = 4.3939e-01, PNorm = 88.3180, GNorm = 43.1226, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.393744
Epoch 215
Validation binary_cross_entropy = 0.452828
Epoch 216
Validation binary_cross_entropy = 0.497375
Epoch 217
Validation binary_cross_entropy = 0.494970
Epoch 218
Validation binary_cross_entropy = 0.469357
Epoch 219
Loss = 2.5637e-01, PNorm = 88.4287, GNorm = 26.7758, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.451778
Epoch 220
Validation binary_cross_entropy = 0.429957
Epoch 221
Validation binary_cross_entropy = 0.408831
Epoch 222
Validation binary_cross_entropy = 0.385440
Epoch 223
Validation binary_cross_entropy = 0.361416
Epoch 224
Loss = 6.7780e-01, PNorm = 88.5349, GNorm = 49.3185, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.351421
Epoch 225
Validation binary_cross_entropy = 0.345748
Epoch 226
Validation binary_cross_entropy = 0.351987
Epoch 227
Validation binary_cross_entropy = 0.359367
Epoch 228
Validation binary_cross_entropy = 0.367966
Epoch 229
Loss = 2.2503e-01, PNorm = 88.6552, GNorm = 18.9529, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.350559
Epoch 230
Validation binary_cross_entropy = 0.330003
Epoch 231
Validation binary_cross_entropy = 0.319379
Epoch 232
Validation binary_cross_entropy = 0.324700
Epoch 233
Validation binary_cross_entropy = 0.341006
Epoch 234
Loss = 2.3843e-01, PNorm = 88.7699, GNorm = 22.6811, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.365715
Epoch 235
Validation binary_cross_entropy = 0.408941
Epoch 236
Validation binary_cross_entropy = 0.443834
Epoch 237
Validation binary_cross_entropy = 0.435789
Epoch 238
Validation binary_cross_entropy = 0.362786
Epoch 239
Loss = 2.1605e-01, PNorm = 88.8491, GNorm = 33.1075, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.315878
Epoch 240
Validation binary_cross_entropy = 0.303540
Epoch 241
Validation binary_cross_entropy = 0.307643
Epoch 242
Validation binary_cross_entropy = 0.353532
Epoch 243
Validation binary_cross_entropy = 0.407171
Epoch 244
Loss = 2.8716e-01, PNorm = 88.9963, GNorm = 36.1129, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.443367
Epoch 245
Validation binary_cross_entropy = 0.439261
Epoch 246
Validation binary_cross_entropy = 0.415541
Epoch 247
Validation binary_cross_entropy = 0.385582
Epoch 248
Validation binary_cross_entropy = 0.358162
Epoch 249
Loss = 2.2661e-01, PNorm = 89.1012, GNorm = 35.1462, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.337754
Epoch 250
Validation binary_cross_entropy = 0.336255
Epoch 251
Validation binary_cross_entropy = 0.342086
Epoch 252
Validation binary_cross_entropy = 0.346665
Epoch 253
Validation binary_cross_entropy = 0.360940
Epoch 254
Loss = 1.9760e-01, PNorm = 89.2106, GNorm = 21.2763, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.374639
Epoch 255
Validation binary_cross_entropy = 0.385232
Epoch 256
Validation binary_cross_entropy = 0.361405
Epoch 257
Validation binary_cross_entropy = 0.336806
Epoch 258
Validation binary_cross_entropy = 0.332399
Epoch 259
Loss = 3.2526e-01, PNorm = 89.2602, GNorm = 12.6027, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.330783
Epoch 260
Validation binary_cross_entropy = 0.323406
Epoch 261
Validation binary_cross_entropy = 0.325042
Epoch 262
Validation binary_cross_entropy = 0.331553
Epoch 263
Validation binary_cross_entropy = 0.340792
Epoch 264
Loss = 4.7426e-01, PNorm = 89.3273, GNorm = 28.4638, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.343508
Epoch 265
Validation binary_cross_entropy = 0.343962
Epoch 266
Validation binary_cross_entropy = 0.360104
Epoch 267
Validation binary_cross_entropy = 0.357791
Epoch 268
Validation binary_cross_entropy = 0.385646
Epoch 269
Loss = 2.5342e-01, PNorm = 89.4448, GNorm = 26.1327, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.453012
Epoch 270
Validation binary_cross_entropy = 0.502565
Epoch 271
Validation binary_cross_entropy = 0.562687
Epoch 272
Validation binary_cross_entropy = 0.550446
Epoch 273
Validation binary_cross_entropy = 0.415842
Epoch 274
Loss = 2.8926e-01, PNorm = 89.5270, GNorm = 19.9198, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.365990
Epoch 275
Validation binary_cross_entropy = 0.347151
Epoch 276
Validation binary_cross_entropy = 0.351806
Epoch 277
Validation binary_cross_entropy = 0.413957
Epoch 278
Validation binary_cross_entropy = 0.464142
Epoch 279
Loss = 2.7021e-01, PNorm = 89.6374, GNorm = 17.9373, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.469287
Epoch 280
Validation binary_cross_entropy = 0.449448
Epoch 281
Validation binary_cross_entropy = 0.435557
Epoch 282
Validation binary_cross_entropy = 0.422784
Epoch 283
Validation binary_cross_entropy = 0.416916
Epoch 284
Loss = 3.2481e-01, PNorm = 89.7264, GNorm = 21.2689, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.426948
Epoch 285
Validation binary_cross_entropy = 0.438357
Epoch 286
Validation binary_cross_entropy = 0.420114
Epoch 287
Validation binary_cross_entropy = 0.393865
Epoch 288
Validation binary_cross_entropy = 0.375985
Epoch 289
Loss = 2.2408e-01, PNorm = 89.8140, GNorm = 15.0583, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.378037
Epoch 290
Validation binary_cross_entropy = 0.398226
Epoch 291
Validation binary_cross_entropy = 0.407522
Epoch 292
Validation binary_cross_entropy = 0.411454
Epoch 293
Validation binary_cross_entropy = 0.427862
Epoch 294
Loss = 1.5160e-01, PNorm = 89.9104, GNorm = 8.8650, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.409803
Epoch 295
Validation binary_cross_entropy = 0.403961
Epoch 296
Validation binary_cross_entropy = 0.395885
Epoch 297
Validation binary_cross_entropy = 0.372850
Epoch 298
Validation binary_cross_entropy = 0.358768
Epoch 299
Loss = 2.2269e-01, PNorm = 89.9958, GNorm = 34.5449, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.346520
Model 0 best validation binary_cross_entropy = 0.303540 on epoch 240
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.180758
Ensemble test binary_cross_entropy = 0.180758
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 73.547699
Epoch 1
Validation binary_cross_entropy = 6.966882
Epoch 2
Validation binary_cross_entropy = 89.354820
Epoch 3
Validation binary_cross_entropy = 5.189694
Epoch 4
Loss = 4.8593e+01, PNorm = 82.9654, GNorm = 1055.2632, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 6.294544
Epoch 5
Validation binary_cross_entropy = 79.436272
Epoch 6
Validation binary_cross_entropy = 50.258183
Epoch 7
Validation binary_cross_entropy = 6.901496
Epoch 8
Validation binary_cross_entropy = 54.668819
Epoch 9
Loss = 4.2559e+01, PNorm = 83.2392, GNorm = 1107.3239, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 78.133469
Epoch 10
Validation binary_cross_entropy = 6.172036
Epoch 11
Validation binary_cross_entropy = 6.054286
Epoch 12
Validation binary_cross_entropy = 19.596851
Epoch 13
Validation binary_cross_entropy = 7.554896
Epoch 14
Loss = 2.3956e+01, PNorm = 83.5389, GNorm = 462.0092, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 6.416431
Epoch 15
Validation binary_cross_entropy = 10.564660
Epoch 16
Validation binary_cross_entropy = 8.215800
Epoch 17
Validation binary_cross_entropy = 7.063457
Epoch 18
Validation binary_cross_entropy = 7.207452
Epoch 19
Loss = 1.7971e+01, PNorm = 83.7604, GNorm = 345.4749, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 7.541782
Epoch 20
Validation binary_cross_entropy = 9.462339
Epoch 21
Validation binary_cross_entropy = 6.244902
Epoch 22
Validation binary_cross_entropy = 6.234109
Epoch 23
Validation binary_cross_entropy = 21.130976
Epoch 24
Loss = 2.4768e+01, PNorm = 83.9328, GNorm = 963.6498, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 5.457364
Epoch 25
Validation binary_cross_entropy = 5.878468
Epoch 26
Validation binary_cross_entropy = 13.719007
Epoch 27
Validation binary_cross_entropy = 51.549835
Epoch 28
Validation binary_cross_entropy = 12.846863
Epoch 29
Loss = 1.4663e+01, PNorm = 84.0759, GNorm = 396.2197, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.714452
Epoch 30
Validation binary_cross_entropy = 7.644191
Epoch 31
Validation binary_cross_entropy = 16.032465
Epoch 32
Validation binary_cross_entropy = 8.720074
Epoch 33
Validation binary_cross_entropy = 6.722422
Epoch 34
Loss = 1.0141e+01, PNorm = 84.2057, GNorm = 246.1593, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 12.814954
Epoch 35
Validation binary_cross_entropy = 11.286839
Epoch 36
Validation binary_cross_entropy = 5.450733
Epoch 37
Validation binary_cross_entropy = 5.311257
Epoch 38
Validation binary_cross_entropy = 9.123822
Epoch 39
Loss = 1.1432e+01, PNorm = 84.3022, GNorm = 420.2259, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 10.733675
Epoch 40
Validation binary_cross_entropy = 4.902103
Epoch 41
Validation binary_cross_entropy = 5.015900
Epoch 42
Validation binary_cross_entropy = 8.484965
Epoch 43
Validation binary_cross_entropy = 24.382523
Epoch 44
Loss = 9.7874e+00, PNorm = 84.3829, GNorm = 297.7778, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 4.723778
Epoch 45
Validation binary_cross_entropy = 4.275039
Epoch 46
Validation binary_cross_entropy = 3.901191
Epoch 47
Validation binary_cross_entropy = 8.072100
Epoch 48
Validation binary_cross_entropy = 10.191085
Epoch 49
Loss = 8.8661e+00, PNorm = 84.4584, GNorm = 232.9960, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 8.061495
Epoch 50
Validation binary_cross_entropy = 4.080814
Epoch 51
Validation binary_cross_entropy = 4.692235
Epoch 52
Validation binary_cross_entropy = 4.389182
Epoch 53
Validation binary_cross_entropy = 4.057441
Epoch 54
Loss = 7.0079e+00, PNorm = 84.5251, GNorm = 273.0745, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 4.415901
Epoch 55
Validation binary_cross_entropy = 4.391870
Epoch 56
Validation binary_cross_entropy = 4.081637
Epoch 57
Validation binary_cross_entropy = 3.366848
Epoch 58
Validation binary_cross_entropy = 3.823994
Epoch 59
Loss = 1.5830e+00, PNorm = 84.5991, GNorm = 70.1674, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 7.592065
Epoch 60
Validation binary_cross_entropy = 9.821850
Epoch 61
Validation binary_cross_entropy = 5.439147
Epoch 62
Validation binary_cross_entropy = 2.510273
Epoch 63
Validation binary_cross_entropy = 2.635675
Epoch 64
Loss = 3.9687e+00, PNorm = 84.6633, GNorm = 197.1463, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.292027
Epoch 65
Validation binary_cross_entropy = 5.453702
Epoch 66
Validation binary_cross_entropy = 6.830548
Epoch 67
Validation binary_cross_entropy = 3.544534
Epoch 68
Validation binary_cross_entropy = 2.060234
Epoch 69
Loss = 3.9358e+00, PNorm = 84.7467, GNorm = 151.0660, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 1.493562
Epoch 70
Validation binary_cross_entropy = 1.936250
Epoch 71
Validation binary_cross_entropy = 2.856521
Epoch 72
Validation binary_cross_entropy = 3.623801
Epoch 73
Validation binary_cross_entropy = 2.568018
Epoch 74
Loss = 3.2208e+00, PNorm = 84.8324, GNorm = 84.0714, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.182148
Epoch 75
Validation binary_cross_entropy = 1.300086
Epoch 76
Validation binary_cross_entropy = 1.249267
Epoch 77
Validation binary_cross_entropy = 1.168471
Epoch 78
Validation binary_cross_entropy = 1.792124
Epoch 79
Loss = 1.6423e+00, PNorm = 84.9716, GNorm = 103.0432, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.903597
Epoch 80
Validation binary_cross_entropy = 1.897470
Epoch 81
Validation binary_cross_entropy = 1.909385
Epoch 82
Validation binary_cross_entropy = 1.250176
Epoch 83
Validation binary_cross_entropy = 1.236797
Epoch 84
Loss = 1.6029e+00, PNorm = 85.1216, GNorm = 112.3113, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 2.017372
Epoch 85
Validation binary_cross_entropy = 1.978844
Epoch 86
Validation binary_cross_entropy = 1.120837
Epoch 87
Validation binary_cross_entropy = 1.024081
Epoch 88
Validation binary_cross_entropy = 0.994898
Epoch 89
Loss = 3.2477e+00, PNorm = 85.2186, GNorm = 118.1712, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.606730
Epoch 90
Validation binary_cross_entropy = 1.888419
Epoch 91
Validation binary_cross_entropy = 3.218895
Epoch 92
Validation binary_cross_entropy = 3.643410
Epoch 93
Validation binary_cross_entropy = 3.564387
Epoch 94
Loss = 1.8508e+00, PNorm = 85.3211, GNorm = 99.7970, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 3.164227
Epoch 95
Validation binary_cross_entropy = 1.825155
Epoch 96
Validation binary_cross_entropy = 1.524678
Epoch 97
Validation binary_cross_entropy = 0.862045
Epoch 98
Validation binary_cross_entropy = 0.865806
Epoch 99
Loss = 1.4368e+00, PNorm = 85.4043, GNorm = 104.9388, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.541704
Epoch 100
Validation binary_cross_entropy = 1.644134
Epoch 101
Validation binary_cross_entropy = 0.898893
Epoch 102
Validation binary_cross_entropy = 0.896712
Epoch 103
Validation binary_cross_entropy = 0.999430
Epoch 104
Loss = 7.9819e-01, PNorm = 85.4922, GNorm = 46.6169, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.002272
Epoch 105
Validation binary_cross_entropy = 0.920627
Epoch 106
Validation binary_cross_entropy = 0.793271
Epoch 107
Validation binary_cross_entropy = 0.753843
Epoch 108
Validation binary_cross_entropy = 0.742807
Epoch 109
Loss = 1.4688e+00, PNorm = 85.6137, GNorm = 85.0245, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.809110
Epoch 110
Validation binary_cross_entropy = 0.992073
Epoch 111
Validation binary_cross_entropy = 1.670057
Epoch 112
Validation binary_cross_entropy = 0.873705
Epoch 113
Validation binary_cross_entropy = 0.681114
Epoch 114
Loss = 1.7112e+00, PNorm = 85.7778, GNorm = 134.1452, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.647769
Epoch 115
Validation binary_cross_entropy = 0.696991
Epoch 116
Validation binary_cross_entropy = 0.666600
Epoch 117
Validation binary_cross_entropy = 0.820657
Epoch 118
Validation binary_cross_entropy = 1.179643
Epoch 119
Loss = 8.5327e-01, PNorm = 85.9669, GNorm = 83.2863, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.170800
Epoch 120
Validation binary_cross_entropy = 0.866616
Epoch 121
Validation binary_cross_entropy = 0.596052
Epoch 122
Validation binary_cross_entropy = 0.594566
Epoch 123
Validation binary_cross_entropy = 0.582319
Epoch 124
Loss = 1.5295e+00, PNorm = 86.1158, GNorm = 81.8355, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.597030
Epoch 125
Validation binary_cross_entropy = 0.648768
Epoch 126
Validation binary_cross_entropy = 0.655330
Epoch 127
Validation binary_cross_entropy = 0.616966
Epoch 128
Validation binary_cross_entropy = 0.563507
Epoch 129
Loss = 8.5587e-01, PNorm = 86.2380, GNorm = 62.3716, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.517494
Epoch 130
Validation binary_cross_entropy = 0.511156
Epoch 131
Validation binary_cross_entropy = 0.507507
Epoch 132
Validation binary_cross_entropy = 0.510141
Epoch 133
Validation binary_cross_entropy = 0.507561
Epoch 134
Loss = 6.7180e-01, PNorm = 86.3459, GNorm = 60.3928, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.504697
Epoch 135
Validation binary_cross_entropy = 0.505408
Epoch 136
Validation binary_cross_entropy = 0.508609
Epoch 137
Validation binary_cross_entropy = 0.515312
Epoch 138
Validation binary_cross_entropy = 0.531441
Epoch 139
Loss = 9.0383e-01, PNorm = 86.4986, GNorm = 37.5154, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.554565
Epoch 140
Validation binary_cross_entropy = 0.518686
Epoch 141
Validation binary_cross_entropy = 0.489930
Epoch 142
Validation binary_cross_entropy = 0.485808
Epoch 143
Validation binary_cross_entropy = 0.485679
Epoch 144
Loss = 1.0422e+00, PNorm = 86.6532, GNorm = 97.5232, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.488406
Epoch 145
Validation binary_cross_entropy = 0.489064
Epoch 146
Validation binary_cross_entropy = 0.524135
Epoch 147
Validation binary_cross_entropy = 0.584180
Epoch 148
Validation binary_cross_entropy = 0.656977
Epoch 149
Loss = 9.1037e-01, PNorm = 86.7946, GNorm = 73.5160, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.687713
Epoch 150
Validation binary_cross_entropy = 0.673226
Epoch 151
Validation binary_cross_entropy = 0.632069
Epoch 152
Validation binary_cross_entropy = 0.549770
Epoch 153
Validation binary_cross_entropy = 0.531005
Epoch 154
Loss = 9.2856e-01, PNorm = 86.9451, GNorm = 48.8357, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.559521
Epoch 155
Validation binary_cross_entropy = 0.551030
Epoch 156
Validation binary_cross_entropy = 0.521894
Epoch 157
Validation binary_cross_entropy = 0.496257
Epoch 158
Validation binary_cross_entropy = 0.525580
Epoch 159
Loss = 1.4200e+00, PNorm = 87.0790, GNorm = 93.9832, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.583408
Epoch 160
Validation binary_cross_entropy = 0.591525
Epoch 161
Validation binary_cross_entropy = 0.554436
Epoch 162
Validation binary_cross_entropy = 0.521273
Epoch 163
Validation binary_cross_entropy = 0.511152
Epoch 164
Loss = 6.2980e-01, PNorm = 87.2434, GNorm = 67.4917, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.498868
Epoch 165
Validation binary_cross_entropy = 0.465708
Epoch 166
Validation binary_cross_entropy = 0.422835
Epoch 167
Validation binary_cross_entropy = 0.431946
Epoch 168
Validation binary_cross_entropy = 0.451132
Epoch 169
Loss = 7.3118e-01, PNorm = 87.3903, GNorm = 43.3431, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.453997
Epoch 170
Validation binary_cross_entropy = 0.418743
Epoch 171
Validation binary_cross_entropy = 0.392975
Epoch 172
Validation binary_cross_entropy = 0.389692
Epoch 173
Validation binary_cross_entropy = 0.386642
Epoch 174
Loss = 1.0728e+00, PNorm = 87.4899, GNorm = 88.6924, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.382209
Epoch 175
Validation binary_cross_entropy = 0.415052
Epoch 176
Validation binary_cross_entropy = 0.448628
Epoch 177
Validation binary_cross_entropy = 0.452183
Epoch 178
Validation binary_cross_entropy = 0.425942
Epoch 179
Loss = 1.4541e+00, PNorm = 87.6373, GNorm = 74.8104, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.405245
Epoch 180
Validation binary_cross_entropy = 0.383175
Epoch 181
Validation binary_cross_entropy = 0.368753
Epoch 182
Validation binary_cross_entropy = 0.359352
Epoch 183
Validation binary_cross_entropy = 0.362731
Epoch 184
Loss = 7.3313e-01, PNorm = 87.7601, GNorm = 60.2371, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.385175
Epoch 185
Validation binary_cross_entropy = 0.432012
Epoch 186
Validation binary_cross_entropy = 0.476470
Epoch 187
Validation binary_cross_entropy = 0.466445
Epoch 188
Validation binary_cross_entropy = 0.433025
Epoch 189
Loss = 8.0889e-01, PNorm = 87.9086, GNorm = 56.9636, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.415128
Epoch 190
Validation binary_cross_entropy = 0.411999
Epoch 191
Validation binary_cross_entropy = 0.427210
Epoch 192
Validation binary_cross_entropy = 0.414213
Epoch 193
Validation binary_cross_entropy = 0.383149
Epoch 194
Loss = 4.8190e-01, PNorm = 87.9918, GNorm = 42.2450, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.364764
Epoch 195
Validation binary_cross_entropy = 0.371040
Epoch 196
Validation binary_cross_entropy = 0.375669
Epoch 197
Validation binary_cross_entropy = 0.407917
Epoch 198
Validation binary_cross_entropy = 0.480431
Epoch 199
Loss = 5.5316e-01, PNorm = 88.0826, GNorm = 33.6506, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.489782
Epoch 200
Validation binary_cross_entropy = 0.443300
Epoch 201
Validation binary_cross_entropy = 0.421751
Epoch 202
Validation binary_cross_entropy = 0.471823
Epoch 203
Validation binary_cross_entropy = 0.504491
Epoch 204
Loss = 5.4637e-01, PNorm = 88.1700, GNorm = 29.7797, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.483183
Epoch 205
Validation binary_cross_entropy = 0.462568
Epoch 206
Validation binary_cross_entropy = 0.438533
Epoch 207
Validation binary_cross_entropy = 0.416062
Epoch 208
Validation binary_cross_entropy = 0.407541
Epoch 209
Loss = 3.7942e-01, PNorm = 88.2701, GNorm = 36.0881, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.394994
Epoch 210
Validation binary_cross_entropy = 0.383999
Epoch 211
Validation binary_cross_entropy = 0.399054
Epoch 212
Validation binary_cross_entropy = 0.424392
Epoch 213
Validation binary_cross_entropy = 0.415051
Epoch 214
Loss = 4.0701e-01, PNorm = 88.3535, GNorm = 35.9337, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.431447
Epoch 215
Validation binary_cross_entropy = 0.459222
Epoch 216
Validation binary_cross_entropy = 0.461462
Epoch 217
Validation binary_cross_entropy = 0.407798
Epoch 218
Validation binary_cross_entropy = 0.384213
Epoch 219
Loss = 7.1979e-01, PNorm = 88.4291, GNorm = 54.9724, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.361655
Epoch 220
Validation binary_cross_entropy = 0.357242
Epoch 221
Validation binary_cross_entropy = 0.367447
Epoch 222
Validation binary_cross_entropy = 0.376475
Epoch 223
Validation binary_cross_entropy = 0.394779
Epoch 224
Loss = 6.1282e-01, PNorm = 88.4816, GNorm = 36.0248, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.390061
Epoch 225
Validation binary_cross_entropy = 0.402734
Epoch 226
Validation binary_cross_entropy = 0.381156
Epoch 227
Validation binary_cross_entropy = 0.362718
Epoch 228
Validation binary_cross_entropy = 0.358559
Epoch 229
Loss = 5.2432e-01, PNorm = 88.5276, GNorm = 36.7777, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.365510
Epoch 230
Validation binary_cross_entropy = 0.379612
Epoch 231
Validation binary_cross_entropy = 0.386136
Epoch 232
Validation binary_cross_entropy = 0.400740
Epoch 233
Validation binary_cross_entropy = 0.424266
Epoch 234
Loss = 3.4026e-01, PNorm = 88.5994, GNorm = 56.4384, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.427693
Epoch 235
Validation binary_cross_entropy = 0.397608
Epoch 236
Validation binary_cross_entropy = 0.383975
Epoch 237
Validation binary_cross_entropy = 0.384903
Epoch 238
Validation binary_cross_entropy = 0.396269
Epoch 239
Loss = 3.7112e-01, PNorm = 88.6798, GNorm = 33.3536, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.409251
Epoch 240
Validation binary_cross_entropy = 0.418869
Epoch 241
Validation binary_cross_entropy = 0.390774
Epoch 242
Validation binary_cross_entropy = 0.379145
Epoch 243
Validation binary_cross_entropy = 0.368931
Epoch 244
Loss = 5.2642e-01, PNorm = 88.7806, GNorm = 32.6118, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.364112
Epoch 245
Validation binary_cross_entropy = 0.363306
Epoch 246
Validation binary_cross_entropy = 0.366511
Epoch 247
Validation binary_cross_entropy = 0.377979
Epoch 248
Validation binary_cross_entropy = 0.388697
Epoch 249
Loss = 2.4559e-01, PNorm = 88.8720, GNorm = 22.9034, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.400419
Epoch 250
Validation binary_cross_entropy = 0.414377
Epoch 251
Validation binary_cross_entropy = 0.442034
Epoch 252
Validation binary_cross_entropy = 0.482478
Epoch 253
Validation binary_cross_entropy = 0.522104
Epoch 254
Loss = 2.6346e-01, PNorm = 88.9623, GNorm = 37.1698, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.525879
Epoch 255
Validation binary_cross_entropy = 0.489716
Epoch 256
Validation binary_cross_entropy = 0.449809
Epoch 257
Validation binary_cross_entropy = 0.418736
Epoch 258
Validation binary_cross_entropy = 0.400902
Epoch 259
Loss = 5.3806e-01, PNorm = 89.0278, GNorm = 47.1124, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.393947
Epoch 260
Validation binary_cross_entropy = 0.396706
Epoch 261
Validation binary_cross_entropy = 0.412437
Epoch 262
Validation binary_cross_entropy = 0.416280
Epoch 263
Validation binary_cross_entropy = 0.410793
Epoch 264
Loss = 3.5388e-01, PNorm = 89.1313, GNorm = 28.0466, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.405433
Epoch 265
Validation binary_cross_entropy = 0.413741
Epoch 266
Validation binary_cross_entropy = 0.431489
Epoch 267
Validation binary_cross_entropy = 0.419771
Epoch 268
Validation binary_cross_entropy = 0.359668
Epoch 269
Loss = 5.0465e-01, PNorm = 89.2101, GNorm = 27.3508, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.340179
Epoch 270
Validation binary_cross_entropy = 0.337993
Epoch 271
Validation binary_cross_entropy = 0.368539
Epoch 272
Validation binary_cross_entropy = 0.443622
Epoch 273
Validation binary_cross_entropy = 0.467362
Epoch 274
Loss = 4.6456e-01, PNorm = 89.2637, GNorm = 33.2594, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.425435
Epoch 275
Validation binary_cross_entropy = 0.384071
Epoch 276
Validation binary_cross_entropy = 0.377387
Epoch 277
Validation binary_cross_entropy = 0.401635
Epoch 278
Validation binary_cross_entropy = 0.469665
Epoch 279
Loss = 3.4359e-01, PNorm = 89.3326, GNorm = 31.7090, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.509931
Epoch 280
Validation binary_cross_entropy = 0.508249
Epoch 281
Validation binary_cross_entropy = 0.455036
Epoch 282
Validation binary_cross_entropy = 0.410297
Epoch 283
Validation binary_cross_entropy = 0.382773
Epoch 284
Loss = 2.9173e-01, PNorm = 89.4181, GNorm = 22.4747, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.376616
Epoch 285
Validation binary_cross_entropy = 0.380603
Epoch 286
Validation binary_cross_entropy = 0.386831
Epoch 287
Validation binary_cross_entropy = 0.427245
Epoch 288
Validation binary_cross_entropy = 0.496497
Epoch 289
Loss = 3.2578e-01, PNorm = 89.5120, GNorm = 22.1356, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.497999
Epoch 290
Validation binary_cross_entropy = 0.412671
Epoch 291
Validation binary_cross_entropy = 0.359146
Epoch 292
Validation binary_cross_entropy = 0.358491
Epoch 293
Validation binary_cross_entropy = 0.363138
Epoch 294
Loss = 4.9043e-01, PNorm = 89.5572, GNorm = 36.8052, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.373777
Epoch 295
Validation binary_cross_entropy = 0.401597
Epoch 296
Validation binary_cross_entropy = 0.467488
Epoch 297
Validation binary_cross_entropy = 0.499736
Epoch 298
Validation binary_cross_entropy = 0.454172
Epoch 299
Loss = 3.7311e-01, PNorm = 89.6151, GNorm = 20.3182, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.412708
Model 0 best validation binary_cross_entropy = 0.337993 on epoch 270
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.182712
Ensemble test binary_cross_entropy = 0.182712
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 23.595459
Epoch 1
Validation binary_cross_entropy = 3.259543
Epoch 2
Validation binary_cross_entropy = 4.622611
Epoch 3
Validation binary_cross_entropy = 33.388969
Epoch 4
Loss = 3.3504e+01, PNorm = 82.9707, GNorm = 957.6522, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 11.964976
Epoch 5
Validation binary_cross_entropy = 6.007349
Epoch 6
Validation binary_cross_entropy = 70.200645
Epoch 7
Validation binary_cross_entropy = 6.200041
Epoch 8
Validation binary_cross_entropy = 73.793930
Epoch 9
Loss = 3.9720e+01, PNorm = 83.2573, GNorm = 1115.7649, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 12.365930
Epoch 10
Validation binary_cross_entropy = 11.639913
Epoch 11
Validation binary_cross_entropy = 38.564148
Epoch 12
Validation binary_cross_entropy = 9.968169
Epoch 13
Validation binary_cross_entropy = 7.985635
Epoch 14
Loss = 2.1655e+01, PNorm = 83.5925, GNorm = 579.7814, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 29.672880
Epoch 15
Validation binary_cross_entropy = 7.857734
Epoch 16
Validation binary_cross_entropy = 17.825239
Epoch 17
Validation binary_cross_entropy = 13.618018
Epoch 18
Validation binary_cross_entropy = 9.960951
Epoch 19
Loss = 1.2427e+01, PNorm = 83.8358, GNorm = 216.4023, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 16.665302
Epoch 20
Validation binary_cross_entropy = 24.887615
Epoch 21
Validation binary_cross_entropy = 10.245203
Epoch 22
Validation binary_cross_entropy = 6.929176
Epoch 23
Validation binary_cross_entropy = 12.877194
Epoch 24
Loss = 2.3186e+01, PNorm = 84.0366, GNorm = 629.3583, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 21.964653
Epoch 25
Validation binary_cross_entropy = 6.302605
Epoch 26
Validation binary_cross_entropy = 6.447835
Epoch 27
Validation binary_cross_entropy = 9.103362
Epoch 28
Validation binary_cross_entropy = 18.106062
Epoch 29
Loss = 1.7503e+01, PNorm = 84.1746, GNorm = 449.5236, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.307066
Epoch 30
Validation binary_cross_entropy = 7.240486
Epoch 31
Validation binary_cross_entropy = 9.967587
Epoch 32
Validation binary_cross_entropy = 20.071955
Epoch 33
Validation binary_cross_entropy = 9.044519
Epoch 34
Loss = 9.2711e+00, PNorm = 84.2988, GNorm = 278.3112, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 7.634043
Epoch 35
Validation binary_cross_entropy = 7.513530
Epoch 36
Validation binary_cross_entropy = 7.821049
Epoch 37
Validation binary_cross_entropy = 10.503902
Epoch 38
Validation binary_cross_entropy = 7.233604
Epoch 39
Loss = 9.6515e+00, PNorm = 84.3834, GNorm = 502.0474, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 6.908051
Epoch 40
Validation binary_cross_entropy = 5.608489
Epoch 41
Validation binary_cross_entropy = 5.069795
Epoch 42
Validation binary_cross_entropy = 5.613536
Epoch 43
Validation binary_cross_entropy = 7.066745
Epoch 44
Loss = 1.0532e+01, PNorm = 84.4469, GNorm = 374.6650, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 5.078532
Epoch 45
Validation binary_cross_entropy = 3.923511
Epoch 46
Validation binary_cross_entropy = 6.155469
Epoch 47
Validation binary_cross_entropy = 8.703060
Epoch 48
Validation binary_cross_entropy = 8.569529
Epoch 49
Loss = 6.3269e+00, PNorm = 84.5099, GNorm = 310.8886, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 5.110194
Epoch 50
Validation binary_cross_entropy = 3.687664
Epoch 51
Validation binary_cross_entropy = 3.428331
Epoch 52
Validation binary_cross_entropy = 3.703339
Epoch 53
Validation binary_cross_entropy = 7.638587
Epoch 54
Loss = 8.0943e+00, PNorm = 84.5835, GNorm = 167.3591, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 4.781291
Epoch 55
Validation binary_cross_entropy = 3.057950
Epoch 56
Validation binary_cross_entropy = 2.267820
Epoch 57
Validation binary_cross_entropy = 2.724715
Epoch 58
Validation binary_cross_entropy = 3.463105
Epoch 59
Loss = 3.0217e+00, PNorm = 84.6709, GNorm = 106.7703, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 2.856507
Epoch 60
Validation binary_cross_entropy = 2.571549
Epoch 61
Validation binary_cross_entropy = 2.670833
Epoch 62
Validation binary_cross_entropy = 2.407238
Epoch 63
Validation binary_cross_entropy = 5.118849
Epoch 64
Loss = 1.7963e+00, PNorm = 84.7623, GNorm = 83.7036, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 3.415351
Epoch 65
Validation binary_cross_entropy = 2.235867
Epoch 66
Validation binary_cross_entropy = 2.514713
Epoch 67
Validation binary_cross_entropy = 2.996079
Epoch 68
Validation binary_cross_entropy = 2.756370
Epoch 69
Loss = 4.4546e+00, PNorm = 84.8515, GNorm = 161.7355, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 3.147641
Epoch 70
Validation binary_cross_entropy = 4.532171
Epoch 71
Validation binary_cross_entropy = 3.517853
Epoch 72
Validation binary_cross_entropy = 1.880021
Epoch 73
Validation binary_cross_entropy = 1.315998
Epoch 74
Loss = 4.0877e+00, PNorm = 84.9540, GNorm = 131.4092, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.747625
Epoch 75
Validation binary_cross_entropy = 2.918319
Epoch 76
Validation binary_cross_entropy = 3.831384
Epoch 77
Validation binary_cross_entropy = 2.831719
Epoch 78
Validation binary_cross_entropy = 1.487433
Epoch 79
Loss = 2.2316e+00, PNorm = 85.0670, GNorm = 144.8434, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.922103
Epoch 80
Validation binary_cross_entropy = 0.847026
Epoch 81
Validation binary_cross_entropy = 0.854714
Epoch 82
Validation binary_cross_entropy = 1.015703
Epoch 83
Validation binary_cross_entropy = 0.875123
Epoch 84
Loss = 3.4064e+00, PNorm = 85.2011, GNorm = 146.9044, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.795289
Epoch 85
Validation binary_cross_entropy = 0.839944
Epoch 86
Validation binary_cross_entropy = 1.017944
Epoch 87
Validation binary_cross_entropy = 0.669958
Epoch 88
Validation binary_cross_entropy = 0.684035
Epoch 89
Loss = 1.0410e+00, PNorm = 85.3775, GNorm = 124.9155, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.665501
Epoch 90
Validation binary_cross_entropy = 0.812785
Epoch 91
Validation binary_cross_entropy = 0.754363
Epoch 92
Validation binary_cross_entropy = 0.767563
Epoch 93
Validation binary_cross_entropy = 1.057769
Epoch 94
Loss = 1.5764e+00, PNorm = 85.5176, GNorm = 155.7515, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.039656
Epoch 95
Validation binary_cross_entropy = 0.784016
Epoch 96
Validation binary_cross_entropy = 0.799223
Epoch 97
Validation binary_cross_entropy = 1.491382
Epoch 98
Validation binary_cross_entropy = 0.747531
Epoch 99
Loss = 1.7666e+00, PNorm = 85.6663, GNorm = 125.4407, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.766539
Epoch 100
Validation binary_cross_entropy = 0.973348
Epoch 101
Validation binary_cross_entropy = 0.930996
Epoch 102
Validation binary_cross_entropy = 0.761694
Epoch 103
Validation binary_cross_entropy = 0.720338
Epoch 104
Loss = 1.3649e+00, PNorm = 85.7860, GNorm = 125.3706, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.761535
Epoch 105
Validation binary_cross_entropy = 0.741288
Epoch 106
Validation binary_cross_entropy = 0.645988
Epoch 107
Validation binary_cross_entropy = 0.592256
Epoch 108
Validation binary_cross_entropy = 0.572827
Epoch 109
Loss = 1.3180e+00, PNorm = 85.8749, GNorm = 115.0134, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.549804
Epoch 110
Validation binary_cross_entropy = 0.531515
Epoch 111
Validation binary_cross_entropy = 0.508779
Epoch 112
Validation binary_cross_entropy = 0.482658
Epoch 113
Validation binary_cross_entropy = 0.467092
Epoch 114
Loss = 1.7078e+00, PNorm = 85.9681, GNorm = 92.0602, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.456478
Epoch 115
Validation binary_cross_entropy = 0.466916
Epoch 116
Validation binary_cross_entropy = 0.462528
Epoch 117
Validation binary_cross_entropy = 0.449033
Epoch 118
Validation binary_cross_entropy = 0.476095
Epoch 119
Loss = 1.7636e+00, PNorm = 86.0943, GNorm = 93.9762, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.696040
Epoch 120
Validation binary_cross_entropy = 1.000188
Epoch 121
Validation binary_cross_entropy = 0.725196
Epoch 122
Validation binary_cross_entropy = 0.605927
Epoch 123
Validation binary_cross_entropy = 0.480966
Epoch 124
Loss = 1.1600e+00, PNorm = 86.1705, GNorm = 51.0538, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.481471
Epoch 125
Validation binary_cross_entropy = 0.585745
Epoch 126
Validation binary_cross_entropy = 0.643590
Epoch 127
Validation binary_cross_entropy = 0.483615
Epoch 128
Validation binary_cross_entropy = 0.400984
Epoch 129
Loss = 6.5226e-01, PNorm = 86.2672, GNorm = 22.5772, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.397723
Epoch 130
Validation binary_cross_entropy = 0.405105
Epoch 131
Validation binary_cross_entropy = 0.406069
Epoch 132
Validation binary_cross_entropy = 0.407756
Epoch 133
Validation binary_cross_entropy = 0.423697
Epoch 134
Loss = 6.2230e-01, PNorm = 86.3588, GNorm = 60.5184, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.470355
Epoch 135
Validation binary_cross_entropy = 0.412813
Epoch 136
Validation binary_cross_entropy = 0.376078
Epoch 137
Validation binary_cross_entropy = 0.384030
Epoch 138
Validation binary_cross_entropy = 0.371208
Epoch 139
Loss = 8.4211e-01, PNorm = 86.4255, GNorm = 81.0756, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.354989
Epoch 140
Validation binary_cross_entropy = 0.417741
Epoch 141
Validation binary_cross_entropy = 0.513305
Epoch 142
Validation binary_cross_entropy = 0.504101
Epoch 143
Validation binary_cross_entropy = 0.453848
Epoch 144
Loss = 1.3752e+00, PNorm = 86.5172, GNorm = 70.1684, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.376171
Epoch 145
Validation binary_cross_entropy = 0.349996
Epoch 146
Validation binary_cross_entropy = 0.351149
Epoch 147
Validation binary_cross_entropy = 0.380641
Epoch 148
Validation binary_cross_entropy = 0.408460
Epoch 149
Loss = 9.4376e-01, PNorm = 86.6492, GNorm = 52.5620, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.464343
Epoch 150
Validation binary_cross_entropy = 0.455487
Epoch 151
Validation binary_cross_entropy = 0.464752
Epoch 152
Validation binary_cross_entropy = 0.374852
Epoch 153
Validation binary_cross_entropy = 0.336121
Epoch 154
Loss = 4.3994e-01, PNorm = 86.7508, GNorm = 54.3108, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.340547
Epoch 155
Validation binary_cross_entropy = 0.356713
Epoch 156
Validation binary_cross_entropy = 0.381105
Epoch 157
Validation binary_cross_entropy = 0.410620
Epoch 158
Validation binary_cross_entropy = 0.389123
Epoch 159
Loss = 8.1838e-01, PNorm = 86.8415, GNorm = 51.2695, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.392422
Epoch 160
Validation binary_cross_entropy = 0.430025
Epoch 161
Validation binary_cross_entropy = 0.486898
Epoch 162
Validation binary_cross_entropy = 0.474906
Epoch 163
Validation binary_cross_entropy = 0.401191
Epoch 164
Loss = 6.4501e-01, PNorm = 86.9418, GNorm = 40.2465, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.368103
Epoch 165
Validation binary_cross_entropy = 0.359276
Epoch 166
Validation binary_cross_entropy = 0.329400
Epoch 167
Validation binary_cross_entropy = 0.311246
Epoch 168
Validation binary_cross_entropy = 0.311183
Epoch 169
Loss = 3.4377e-01, PNorm = 87.0393, GNorm = 37.8234, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.341102
Epoch 170
Validation binary_cross_entropy = 0.351413
Epoch 171
Validation binary_cross_entropy = 0.361412
Epoch 172
Validation binary_cross_entropy = 0.357142
Epoch 173
Validation binary_cross_entropy = 0.334549
Epoch 174
Loss = 8.0429e-01, PNorm = 87.1443, GNorm = 32.4319, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.291242
Epoch 175
Validation binary_cross_entropy = 0.294999
Epoch 176
Validation binary_cross_entropy = 0.294281
Epoch 177
Validation binary_cross_entropy = 0.292727
Epoch 178
Validation binary_cross_entropy = 0.296433
Epoch 179
Loss = 6.4477e-01, PNorm = 87.2593, GNorm = 54.6217, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.304278
Epoch 180
Validation binary_cross_entropy = 0.313022
Epoch 181
Validation binary_cross_entropy = 0.340369
Epoch 182
Validation binary_cross_entropy = 0.435145
Epoch 183
Validation binary_cross_entropy = 0.491394
Epoch 184
Loss = 1.0523e+00, PNorm = 87.3890, GNorm = 63.5221, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.380858
Epoch 185
Validation binary_cross_entropy = 0.332031
Epoch 186
Validation binary_cross_entropy = 0.316457
Epoch 187
Validation binary_cross_entropy = 0.321385
Epoch 188
Validation binary_cross_entropy = 0.343770
Epoch 189
Loss = 5.3783e-01, PNorm = 87.5251, GNorm = 43.0007, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.362117
Epoch 190
Validation binary_cross_entropy = 0.364077
Epoch 191
Validation binary_cross_entropy = 0.352713
Epoch 192
Validation binary_cross_entropy = 0.345178
Epoch 193
Validation binary_cross_entropy = 0.327316
Epoch 194
Loss = 5.7415e-01, PNorm = 87.6489, GNorm = 31.6027, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.327850
Epoch 195
Validation binary_cross_entropy = 0.359948
Epoch 196
Validation binary_cross_entropy = 0.412351
Epoch 197
Validation binary_cross_entropy = 0.470128
Epoch 198
Validation binary_cross_entropy = 0.413722
Epoch 199
Loss = 1.0164e+00, PNorm = 87.7758, GNorm = 57.2579, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.348161
Epoch 200
Validation binary_cross_entropy = 0.333165
Epoch 201
Validation binary_cross_entropy = 0.327284
Epoch 202
Validation binary_cross_entropy = 0.355797
Epoch 203
Validation binary_cross_entropy = 0.380668
Epoch 204
Loss = 8.4131e-01, PNorm = 87.9333, GNorm = 69.9427, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.371171
Epoch 205
Validation binary_cross_entropy = 0.337368
Epoch 206
Validation binary_cross_entropy = 0.324639
Epoch 207
Validation binary_cross_entropy = 0.321376
Epoch 208
Validation binary_cross_entropy = 0.327731
Epoch 209
Loss = 4.0972e-01, PNorm = 88.0372, GNorm = 25.8859, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.350683
Epoch 210
Validation binary_cross_entropy = 0.406885
Epoch 211
Validation binary_cross_entropy = 0.470893
Epoch 212
Validation binary_cross_entropy = 0.529927
Epoch 213
Validation binary_cross_entropy = 0.558670
Epoch 214
Loss = 6.4835e-01, PNorm = 88.2098, GNorm = 38.6230, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.448420
Epoch 215
Validation binary_cross_entropy = 0.354693
Epoch 216
Validation binary_cross_entropy = 0.327325
Epoch 217
Validation binary_cross_entropy = 0.381699
Epoch 218
Validation binary_cross_entropy = 0.447063
Epoch 219
Loss = 2.9484e-01, PNorm = 88.3408, GNorm = 27.6255, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.467775
Epoch 220
Validation binary_cross_entropy = 0.414705
Epoch 221
Validation binary_cross_entropy = 0.390015
Epoch 222
Validation binary_cross_entropy = 0.379011
Epoch 223
Validation binary_cross_entropy = 0.349886
Epoch 224
Loss = 5.6069e-01, PNorm = 88.4263, GNorm = 30.8085, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.322651
Epoch 225
Validation binary_cross_entropy = 0.313501
Epoch 226
Validation binary_cross_entropy = 0.309955
Epoch 227
Validation binary_cross_entropy = 0.313211
Epoch 228
Validation binary_cross_entropy = 0.325481
Epoch 229
Loss = 3.4912e-01, PNorm = 88.4921, GNorm = 24.6531, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.349780
Epoch 230
Validation binary_cross_entropy = 0.357909
Epoch 231
Validation binary_cross_entropy = 0.336733
Epoch 232
Validation binary_cross_entropy = 0.338331
Epoch 233
Validation binary_cross_entropy = 0.344880
Epoch 234
Loss = 4.5370e-01, PNorm = 88.5936, GNorm = 31.4075, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.354631
Epoch 235
Validation binary_cross_entropy = 0.375241
Epoch 236
Validation binary_cross_entropy = 0.364079
Epoch 237
Validation binary_cross_entropy = 0.342788
Epoch 238
Validation binary_cross_entropy = 0.336507
Epoch 239
Loss = 8.4200e-01, PNorm = 88.7022, GNorm = 70.5228, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.339990
Epoch 240
Validation binary_cross_entropy = 0.355248
Epoch 241
Validation binary_cross_entropy = 0.376277
Epoch 242
Validation binary_cross_entropy = 0.388666
Epoch 243
Validation binary_cross_entropy = 0.375669
Epoch 244
Loss = 4.0750e-01, PNorm = 88.8336, GNorm = 22.4991, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.346208
Epoch 245
Validation binary_cross_entropy = 0.331925
Epoch 246
Validation binary_cross_entropy = 0.347126
Epoch 247
Validation binary_cross_entropy = 0.391503
Epoch 248
Validation binary_cross_entropy = 0.433547
Epoch 249
Loss = 4.0196e-01, PNorm = 88.9389, GNorm = 27.1208, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.444092
Epoch 250
Validation binary_cross_entropy = 0.404913
Epoch 251
Validation binary_cross_entropy = 0.368956
Epoch 252
Validation binary_cross_entropy = 0.343345
Epoch 253
Validation binary_cross_entropy = 0.338623
Epoch 254
Loss = 4.1429e-01, PNorm = 89.0187, GNorm = 31.8307, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.343342
Epoch 255
Validation binary_cross_entropy = 0.355519
Epoch 256
Validation binary_cross_entropy = 0.385645
Epoch 257
Validation binary_cross_entropy = 0.422567
Epoch 258
Validation binary_cross_entropy = 0.444587
Epoch 259
Loss = 3.0371e-01, PNorm = 89.1322, GNorm = 25.6147, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.434769
Epoch 260
Validation binary_cross_entropy = 0.409701
Epoch 261
Validation binary_cross_entropy = 0.371512
Epoch 262
Validation binary_cross_entropy = 0.369955
Epoch 263
Validation binary_cross_entropy = 0.391577
Epoch 264
Loss = 2.0750e-01, PNorm = 89.2462, GNorm = 25.4958, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.409481
Epoch 265
Validation binary_cross_entropy = 0.411283
Epoch 266
Validation binary_cross_entropy = 0.341379
Epoch 267
Validation binary_cross_entropy = 0.316595
Epoch 268
Validation binary_cross_entropy = 0.335589
Epoch 269
Loss = 4.2218e-01, PNorm = 89.3704, GNorm = 27.3495, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.378979
Epoch 270
Validation binary_cross_entropy = 0.418284
Epoch 271
Validation binary_cross_entropy = 0.452369
Epoch 272
Validation binary_cross_entropy = 0.465383
Epoch 273
Validation binary_cross_entropy = 0.436050
Epoch 274
Loss = 3.1218e-01, PNorm = 89.5032, GNorm = 24.3588, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.413792
Epoch 275
Validation binary_cross_entropy = 0.383971
Epoch 276
Validation binary_cross_entropy = 0.385644
Epoch 277
Validation binary_cross_entropy = 0.398058
Epoch 278
Validation binary_cross_entropy = 0.437871
Epoch 279
Loss = 4.9472e-01, PNorm = 89.6414, GNorm = 46.0915, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.432785
Epoch 280
Validation binary_cross_entropy = 0.415358
Epoch 281
Validation binary_cross_entropy = 0.394625
Epoch 282
Validation binary_cross_entropy = 0.409483
Epoch 283
Validation binary_cross_entropy = 0.439087
Epoch 284
Loss = 4.2686e-01, PNorm = 89.7382, GNorm = 39.4855, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.428004
Epoch 285
Validation binary_cross_entropy = 0.383763
Epoch 286
Validation binary_cross_entropy = 0.354891
Epoch 287
Validation binary_cross_entropy = 0.338678
Epoch 288
Validation binary_cross_entropy = 0.333585
Epoch 289
Loss = 1.7838e-01, PNorm = 89.8451, GNorm = 20.1048, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.342631
Epoch 290
Validation binary_cross_entropy = 0.348297
Epoch 291
Validation binary_cross_entropy = 0.347897
Epoch 292
Validation binary_cross_entropy = 0.349135
Epoch 293
Validation binary_cross_entropy = 0.339980
Epoch 294
Loss = 2.8485e-01, PNorm = 89.9474, GNorm = 19.5366, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.330045
Epoch 295
Validation binary_cross_entropy = 0.327693
Epoch 296
Validation binary_cross_entropy = 0.342311
Epoch 297
Validation binary_cross_entropy = 0.372250
Epoch 298
Validation binary_cross_entropy = 0.380994
Epoch 299
Loss = 4.9128e-01, PNorm = 90.0768, GNorm = 23.3286, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.357848
Model 0 best validation binary_cross_entropy = 0.291242 on epoch 174
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.179068
Ensemble test binary_cross_entropy = 0.179068
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 17.112225
Epoch 1
Validation binary_cross_entropy = 3.253201
Epoch 2
Validation binary_cross_entropy = 55.221661
Epoch 3
Validation binary_cross_entropy = 5.604319
Epoch 4
Loss = 4.8581e+01, PNorm = 82.9747, GNorm = 750.3154, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 62.061954
Epoch 5
Validation binary_cross_entropy = 5.368408
Epoch 6
Validation binary_cross_entropy = 24.788317
Epoch 7
Validation binary_cross_entropy = 9.750717
Epoch 8
Validation binary_cross_entropy = 21.022188
Epoch 9
Loss = 3.2383e+01, PNorm = 83.2642, GNorm = 434.5420, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 59.043522
Epoch 10
Validation binary_cross_entropy = 39.772079
Epoch 11
Validation binary_cross_entropy = 24.683924
Epoch 12
Validation binary_cross_entropy = 14.629494
Epoch 13
Validation binary_cross_entropy = 13.860596
Epoch 14
Loss = 2.1596e+01, PNorm = 83.6056, GNorm = 656.8722, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 11.698604
Epoch 15
Validation binary_cross_entropy = 36.114845
Epoch 16
Validation binary_cross_entropy = 21.672773
Epoch 17
Validation binary_cross_entropy = 17.271044
Epoch 18
Validation binary_cross_entropy = 46.968197
Epoch 19
Loss = 3.7870e+01, PNorm = 83.8619, GNorm = 1194.5300, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 25.865919
Epoch 20
Validation binary_cross_entropy = 7.500687
Epoch 21
Validation binary_cross_entropy = 36.335537
Epoch 22
Validation binary_cross_entropy = 60.143188
Epoch 23
Validation binary_cross_entropy = 6.939360
Epoch 24
Loss = 4.2414e+01, PNorm = 84.0527, GNorm = 1031.7573, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.457129
Epoch 25
Validation binary_cross_entropy = 32.277489
Epoch 26
Validation binary_cross_entropy = 39.092022
Epoch 27
Validation binary_cross_entropy = 6.788705
Epoch 28
Validation binary_cross_entropy = 6.684668
Epoch 29
Loss = 3.5426e+01, PNorm = 84.1975, GNorm = 633.1277, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.217845
Epoch 30
Validation binary_cross_entropy = 25.331886
Epoch 31
Validation binary_cross_entropy = 26.681063
Epoch 32
Validation binary_cross_entropy = 6.365555
Epoch 33
Validation binary_cross_entropy = 6.116997
Epoch 34
Loss = 1.8134e+01, PNorm = 84.3002, GNorm = 347.1841, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 5.531735
Epoch 35
Validation binary_cross_entropy = 10.200148
Epoch 36
Validation binary_cross_entropy = 27.925474
Epoch 37
Validation binary_cross_entropy = 19.260107
Epoch 38
Validation binary_cross_entropy = 7.267158
Epoch 39
Loss = 1.0013e+01, PNorm = 84.3771, GNorm = 245.3582, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.719943
Epoch 40
Validation binary_cross_entropy = 4.367699
Epoch 41
Validation binary_cross_entropy = 9.743228
Epoch 42
Validation binary_cross_entropy = 12.538636
Epoch 43
Validation binary_cross_entropy = 10.096453
Epoch 44
Loss = 1.0720e+01, PNorm = 84.4486, GNorm = 400.3112, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 4.607047
Epoch 45
Validation binary_cross_entropy = 3.739330
Epoch 46
Validation binary_cross_entropy = 3.706943
Epoch 47
Validation binary_cross_entropy = 6.700715
Epoch 48
Validation binary_cross_entropy = 6.792100
Epoch 49
Loss = 7.2630e+00, PNorm = 84.5180, GNorm = 259.5342, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.287261
Epoch 50
Validation binary_cross_entropy = 3.734595
Epoch 51
Validation binary_cross_entropy = 3.299498
Epoch 52
Validation binary_cross_entropy = 2.959540
Epoch 53
Validation binary_cross_entropy = 2.829921
Epoch 54
Loss = 2.9264e+00, PNorm = 84.6149, GNorm = 172.4140, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 2.687856
Epoch 55
Validation binary_cross_entropy = 2.766898
Epoch 56
Validation binary_cross_entropy = 2.934639
Epoch 57
Validation binary_cross_entropy = 2.882756
Epoch 58
Validation binary_cross_entropy = 2.670268
Epoch 59
Loss = 6.8033e+00, PNorm = 84.7651, GNorm = 187.5892, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 3.770595
Epoch 60
Validation binary_cross_entropy = 3.049561
Epoch 61
Validation binary_cross_entropy = 2.588224
Epoch 62
Validation binary_cross_entropy = 2.660659
Epoch 63
Validation binary_cross_entropy = 2.540761
Epoch 64
Loss = 4.4370e+00, PNorm = 84.9281, GNorm = 236.2697, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.571021
Epoch 65
Validation binary_cross_entropy = 2.134834
Epoch 66
Validation binary_cross_entropy = 1.773700
Epoch 67
Validation binary_cross_entropy = 1.537628
Epoch 68
Validation binary_cross_entropy = 1.814028
Epoch 69
Loss = 1.4488e+00, PNorm = 85.1436, GNorm = 107.5062, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.311844
Epoch 70
Validation binary_cross_entropy = 2.269664
Epoch 71
Validation binary_cross_entropy = 2.316822
Epoch 72
Validation binary_cross_entropy = 2.382280
Epoch 73
Validation binary_cross_entropy = 2.212513
Epoch 74
Loss = 2.4205e+00, PNorm = 85.3863, GNorm = 124.1445, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.547632
Epoch 75
Validation binary_cross_entropy = 2.599050
Epoch 76
Validation binary_cross_entropy = 2.188523
Epoch 77
Validation binary_cross_entropy = 1.626991
Epoch 78
Validation binary_cross_entropy = 1.580626
Epoch 79
Loss = 1.3980e+00, PNorm = 85.5479, GNorm = 124.9725, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 1.627500
Epoch 80
Validation binary_cross_entropy = 0.935619
Epoch 81
Validation binary_cross_entropy = 0.980548
Epoch 82
Validation binary_cross_entropy = 0.925069
Epoch 83
Validation binary_cross_entropy = 0.933240
Epoch 84
Loss = 1.8831e+00, PNorm = 85.7417, GNorm = 152.3850, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.180328
Epoch 85
Validation binary_cross_entropy = 2.731859
Epoch 86
Validation binary_cross_entropy = 0.984522
Epoch 87
Validation binary_cross_entropy = 1.000715
Epoch 88
Validation binary_cross_entropy = 0.975072
Epoch 89
Loss = 1.7241e+00, PNorm = 85.8566, GNorm = 75.7184, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.940771
Epoch 90
Validation binary_cross_entropy = 2.215499
Epoch 91
Validation binary_cross_entropy = 2.566863
Epoch 92
Validation binary_cross_entropy = 2.423534
Epoch 93
Validation binary_cross_entropy = 2.100471
Epoch 94
Loss = 2.1453e+00, PNorm = 86.0362, GNorm = 113.5817, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 2.072047
Epoch 95
Validation binary_cross_entropy = 2.082121
Epoch 96
Validation binary_cross_entropy = 2.016463
Epoch 97
Validation binary_cross_entropy = 1.958233
Epoch 98
Validation binary_cross_entropy = 1.431022
Epoch 99
Loss = 7.4399e-01, PNorm = 86.1666, GNorm = 68.6997, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.873637
Epoch 100
Validation binary_cross_entropy = 1.416351
Epoch 101
Validation binary_cross_entropy = 2.291908
Epoch 102
Validation binary_cross_entropy = 1.591962
Epoch 103
Validation binary_cross_entropy = 1.365663
Epoch 104
Loss = 1.4588e+00, PNorm = 86.2839, GNorm = 88.9365, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.754528
Epoch 105
Validation binary_cross_entropy = 0.794846
Epoch 106
Validation binary_cross_entropy = 1.538219
Epoch 107
Validation binary_cross_entropy = 1.802230
Epoch 108
Validation binary_cross_entropy = 2.772863
Epoch 109
Loss = 2.2635e+00, PNorm = 86.4296, GNorm = 81.1161, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.833675
Epoch 110
Validation binary_cross_entropy = 0.953408
Epoch 111
Validation binary_cross_entropy = 0.727154
Epoch 112
Validation binary_cross_entropy = 0.669123
Epoch 113
Validation binary_cross_entropy = 0.821425
Epoch 114
Loss = 1.4266e+00, PNorm = 86.5832, GNorm = 79.7071, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.722462
Epoch 115
Validation binary_cross_entropy = 1.082519
Epoch 116
Validation binary_cross_entropy = 0.717745
Epoch 117
Validation binary_cross_entropy = 0.658973
Epoch 118
Validation binary_cross_entropy = 0.756311
Epoch 119
Loss = 1.8271e+00, PNorm = 86.7315, GNorm = 36.0386, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.751720
Epoch 120
Validation binary_cross_entropy = 0.711873
Epoch 121
Validation binary_cross_entropy = 0.715257
Epoch 122
Validation binary_cross_entropy = 0.763676
Epoch 123
Validation binary_cross_entropy = 0.762208
Epoch 124
Loss = 1.6586e+00, PNorm = 86.8502, GNorm = 86.3714, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.643547
Epoch 125
Validation binary_cross_entropy = 0.642382
Epoch 126
Validation binary_cross_entropy = 0.815976
Epoch 127
Validation binary_cross_entropy = 1.532786
Epoch 128
Validation binary_cross_entropy = 1.399161
Epoch 129
Loss = 8.4557e-01, PNorm = 86.9554, GNorm = 42.0748, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.375764
Epoch 130
Validation binary_cross_entropy = 0.971133
Epoch 131
Validation binary_cross_entropy = 1.015817
Epoch 132
Validation binary_cross_entropy = 0.942220
Epoch 133
Validation binary_cross_entropy = 0.738284
Epoch 134
Loss = 1.6457e+00, PNorm = 87.1190, GNorm = 68.8688, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.720482
Epoch 135
Validation binary_cross_entropy = 0.910875
Epoch 136
Validation binary_cross_entropy = 1.418659
Epoch 137
Validation binary_cross_entropy = 2.684429
Epoch 138
Validation binary_cross_entropy = 1.897001
Epoch 139
Loss = 5.7473e-01, PNorm = 87.3197, GNorm = 38.4801, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.741503
Epoch 140
Validation binary_cross_entropy = 0.640166
Epoch 141
Validation binary_cross_entropy = 0.739585
Epoch 142
Validation binary_cross_entropy = 0.778375
Epoch 143
Validation binary_cross_entropy = 0.786684
Epoch 144
Loss = 7.6995e-01, PNorm = 87.5052, GNorm = 94.7773, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.612219
Epoch 145
Validation binary_cross_entropy = 0.600650
Epoch 146
Validation binary_cross_entropy = 0.664210
Epoch 147
Validation binary_cross_entropy = 0.736418
Epoch 148
Validation binary_cross_entropy = 0.948012
Epoch 149
Loss = 1.3304e+00, PNorm = 87.6282, GNorm = 103.0336, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.894621
Epoch 150
Validation binary_cross_entropy = 0.740087
Epoch 151
Validation binary_cross_entropy = 0.769155
Epoch 152
Validation binary_cross_entropy = 0.768288
Epoch 153
Validation binary_cross_entropy = 0.658113
Epoch 154
Loss = 9.9784e-01, PNorm = 87.7991, GNorm = 74.9582, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.555496
Epoch 155
Validation binary_cross_entropy = 0.513124
Epoch 156
Validation binary_cross_entropy = 0.519617
Epoch 157
Validation binary_cross_entropy = 0.504390
Epoch 158
Validation binary_cross_entropy = 0.536743
Epoch 159
Loss = 4.7869e-01, PNorm = 87.9448, GNorm = 55.0821, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.648531
Epoch 160
Validation binary_cross_entropy = 0.733604
Epoch 161
Validation binary_cross_entropy = 0.617987
Epoch 162
Validation binary_cross_entropy = 0.580584
Epoch 163
Validation binary_cross_entropy = 0.552952
Epoch 164
Loss = 6.6151e-01, PNorm = 88.0992, GNorm = 77.2666, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.571344
Epoch 165
Validation binary_cross_entropy = 0.683317
Epoch 166
Validation binary_cross_entropy = 0.608133
Epoch 167
Validation binary_cross_entropy = 0.525087
Epoch 168
Validation binary_cross_entropy = 0.507633
Epoch 169
Loss = 6.4408e-01, PNorm = 88.2216, GNorm = 87.2228, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.608864
Epoch 170
Validation binary_cross_entropy = 0.556062
Epoch 171
Validation binary_cross_entropy = 0.589877
Epoch 172
Validation binary_cross_entropy = 0.736141
Epoch 173
Validation binary_cross_entropy = 0.930100
Epoch 174
Loss = 9.2106e-01, PNorm = 88.3670, GNorm = 69.6231, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.747184
Epoch 175
Validation binary_cross_entropy = 0.464177
Epoch 176
Validation binary_cross_entropy = 0.427994
Epoch 177
Validation binary_cross_entropy = 0.435762
Epoch 178
Validation binary_cross_entropy = 0.443833
Epoch 179
Loss = 8.8917e-01, PNorm = 88.5185, GNorm = 47.5888, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.449088
Epoch 180
Validation binary_cross_entropy = 0.463181
Epoch 181
Validation binary_cross_entropy = 0.461526
Epoch 182
Validation binary_cross_entropy = 0.479755
Epoch 183
Validation binary_cross_entropy = 0.504663
Epoch 184
Loss = 5.8080e-01, PNorm = 88.6737, GNorm = 42.1259, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.579044
Epoch 185
Validation binary_cross_entropy = 0.543786
Epoch 186
Validation binary_cross_entropy = 0.433788
Epoch 187
Validation binary_cross_entropy = 0.416379
Epoch 188
Validation binary_cross_entropy = 0.418483
Epoch 189
Loss = 4.9406e-01, PNorm = 88.7684, GNorm = 28.8508, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.449293
Epoch 190
Validation binary_cross_entropy = 0.523395
Epoch 191
Validation binary_cross_entropy = 0.696114
Epoch 192
Validation binary_cross_entropy = 0.839547
Epoch 193
Validation binary_cross_entropy = 0.710131
Epoch 194
Loss = 7.0424e-01, PNorm = 88.8690, GNorm = 78.4390, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.541888
Epoch 195
Validation binary_cross_entropy = 0.493338
Epoch 196
Validation binary_cross_entropy = 0.495755
Epoch 197
Validation binary_cross_entropy = 0.530532
Epoch 198
Validation binary_cross_entropy = 0.594016
Epoch 199
Loss = 8.5619e-01, PNorm = 88.9913, GNorm = 33.5086, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.736728
Epoch 200
Validation binary_cross_entropy = 0.831052
Epoch 201
Validation binary_cross_entropy = 0.891360
Epoch 202
Validation binary_cross_entropy = 0.872639
Epoch 203
Validation binary_cross_entropy = 0.691057
Epoch 204
Loss = 6.4871e-01, PNorm = 89.0904, GNorm = 47.0059, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.560405
Epoch 205
Validation binary_cross_entropy = 0.503626
Epoch 206
Validation binary_cross_entropy = 0.493620
Epoch 207
Validation binary_cross_entropy = 0.522997
Epoch 208
Validation binary_cross_entropy = 0.626690
Epoch 209
Loss = 3.2335e-01, PNorm = 89.2152, GNorm = 36.1424, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.724851
Epoch 210
Validation binary_cross_entropy = 0.737785
Epoch 211
Validation binary_cross_entropy = 0.701882
Epoch 212
Validation binary_cross_entropy = 0.664366
Epoch 213
Validation binary_cross_entropy = 0.617638
Epoch 214
Loss = 4.2661e-01, PNorm = 89.3332, GNorm = 33.7709, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.614189
Epoch 215
Validation binary_cross_entropy = 0.582228
Epoch 216
Validation binary_cross_entropy = 0.541622
Epoch 217
Validation binary_cross_entropy = 0.562571
Epoch 218
Validation binary_cross_entropy = 0.594928
Epoch 219
Loss = 4.3693e-01, PNorm = 89.4274, GNorm = 37.0961, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.582828
Epoch 220
Validation binary_cross_entropy = 0.563108
Epoch 221
Validation binary_cross_entropy = 0.556360
Epoch 222
Validation binary_cross_entropy = 0.567887
Epoch 223
Validation binary_cross_entropy = 0.523411
Epoch 224
Loss = 2.5295e-01, PNorm = 89.5286, GNorm = 20.1569, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.457398
Epoch 225
Validation binary_cross_entropy = 0.437022
Epoch 226
Validation binary_cross_entropy = 0.426046
Epoch 227
Validation binary_cross_entropy = 0.464684
Epoch 228
Validation binary_cross_entropy = 0.537736
Epoch 229
Loss = 6.7153e-01, PNorm = 89.6254, GNorm = 59.9870, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.542664
Epoch 230
Validation binary_cross_entropy = 0.438077
Epoch 231
Validation binary_cross_entropy = 0.400277
Epoch 232
Validation binary_cross_entropy = 0.383266
Epoch 233
Validation binary_cross_entropy = 0.395319
Epoch 234
Loss = 6.5176e-01, PNorm = 89.7548, GNorm = 33.7703, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.485064
Epoch 235
Validation binary_cross_entropy = 0.510104
Epoch 236
Validation binary_cross_entropy = 0.491655
Epoch 237
Validation binary_cross_entropy = 0.441374
Epoch 238
Validation binary_cross_entropy = 0.416925
Epoch 239
Loss = 3.1742e-01, PNorm = 89.8759, GNorm = 22.5692, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.400311
Epoch 240
Validation binary_cross_entropy = 0.385992
Epoch 241
Validation binary_cross_entropy = 0.379764
Epoch 242
Validation binary_cross_entropy = 0.381950
Epoch 243
Validation binary_cross_entropy = 0.399491
Epoch 244
Loss = 3.2119e-01, PNorm = 89.9859, GNorm = 34.6966, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.454205
Epoch 245
Validation binary_cross_entropy = 0.489685
Epoch 246
Validation binary_cross_entropy = 0.428202
Epoch 247
Validation binary_cross_entropy = 0.395540
Epoch 248
Validation binary_cross_entropy = 0.378364
Epoch 249
Loss = 4.9909e-01, PNorm = 90.0679, GNorm = 48.6810, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.377400
Epoch 250
Validation binary_cross_entropy = 0.384532
Epoch 251
Validation binary_cross_entropy = 0.402130
Epoch 252
Validation binary_cross_entropy = 0.408869
Epoch 253
Validation binary_cross_entropy = 0.427593
Epoch 254
Loss = 2.9603e-01, PNorm = 90.1556, GNorm = 18.9608, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.434786
Epoch 255
Validation binary_cross_entropy = 0.438362
Epoch 256
Validation binary_cross_entropy = 0.453637
Epoch 257
Validation binary_cross_entropy = 0.421288
Epoch 258
Validation binary_cross_entropy = 0.398805
Epoch 259
Loss = 3.7471e-01, PNorm = 90.2804, GNorm = 38.2646, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.376898
Epoch 260
Validation binary_cross_entropy = 0.381040
Epoch 261
Validation binary_cross_entropy = 0.382024
Epoch 262
Validation binary_cross_entropy = 0.403468
Epoch 263
Validation binary_cross_entropy = 0.450187
Epoch 264
Loss = 3.0885e-01, PNorm = 90.3995, GNorm = 14.6007, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.452234
Epoch 265
Validation binary_cross_entropy = 0.425494
Epoch 266
Validation binary_cross_entropy = 0.408538
Epoch 267
Validation binary_cross_entropy = 0.388021
Epoch 268
Validation binary_cross_entropy = 0.377008
Epoch 269
Loss = 3.5921e-01, PNorm = 90.4845, GNorm = 45.2560, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.373552
Epoch 270
Validation binary_cross_entropy = 0.378841
Epoch 271
Validation binary_cross_entropy = 0.409649
Epoch 272
Validation binary_cross_entropy = 0.431261
Epoch 273
Validation binary_cross_entropy = 0.436573
Epoch 274
Loss = 1.7926e-01, PNorm = 90.6334, GNorm = 16.3419, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.426634
Epoch 275
Validation binary_cross_entropy = 0.395682
Epoch 276
Validation binary_cross_entropy = 0.371510
Epoch 277
Validation binary_cross_entropy = 0.370320
Epoch 278
Validation binary_cross_entropy = 0.366860
Epoch 279
Loss = 3.2783e-01, PNorm = 90.7548, GNorm = 28.5501, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.376950
Epoch 280
Validation binary_cross_entropy = 0.412760
Epoch 281
Validation binary_cross_entropy = 0.413315
Epoch 282
Validation binary_cross_entropy = 0.408977
Epoch 283
Validation binary_cross_entropy = 0.403096
Epoch 284
Loss = 2.8326e-01, PNorm = 90.8624, GNorm = 21.3781, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.388930
Epoch 285
Validation binary_cross_entropy = 0.390960
Epoch 286
Validation binary_cross_entropy = 0.401835
Epoch 287
Validation binary_cross_entropy = 0.412223
Epoch 288
Validation binary_cross_entropy = 0.417969
Epoch 289
Loss = 2.1696e-01, PNorm = 90.9875, GNorm = 14.7558, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.424347
Epoch 290
Validation binary_cross_entropy = 0.423218
Epoch 291
Validation binary_cross_entropy = 0.417305
Epoch 292
Validation binary_cross_entropy = 0.420867
Epoch 293
Validation binary_cross_entropy = 0.399788
Epoch 294
Loss = 2.4378e-01, PNorm = 91.0596, GNorm = 15.8498, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.383755
Epoch 295
Validation binary_cross_entropy = 0.381963
Epoch 296
Validation binary_cross_entropy = 0.387132
Epoch 297
Validation binary_cross_entropy = 0.395285
Epoch 298
Validation binary_cross_entropy = 0.377035
Epoch 299
Loss = 4.2469e-01, PNorm = 91.1421, GNorm = 32.4455, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.364718
Model 0 best validation binary_cross_entropy = 0.364718 on epoch 299
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.168620
Ensemble test binary_cross_entropy = 0.168620
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 6.101624
Epoch 1
Validation binary_cross_entropy = 21.704363
Epoch 2
Validation binary_cross_entropy = 4.834266
Epoch 3
Validation binary_cross_entropy = 5.147782
Epoch 4
Loss = 3.5999e+01, PNorm = 82.9784, GNorm = 712.0999, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 10.268098
Epoch 5
Validation binary_cross_entropy = 26.015957
Epoch 6
Validation binary_cross_entropy = 12.867446
Epoch 7
Validation binary_cross_entropy = 38.123138
Epoch 8
Validation binary_cross_entropy = 5.586611
Epoch 9
Loss = 5.5069e+01, PNorm = 83.2866, GNorm = 1355.7948, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 21.495768
Epoch 10
Validation binary_cross_entropy = 10.252826
Epoch 11
Validation binary_cross_entropy = 10.318740
Epoch 12
Validation binary_cross_entropy = 17.142050
Epoch 13
Validation binary_cross_entropy = 6.942794
Epoch 14
Loss = 3.6569e+01, PNorm = 83.6521, GNorm = 750.9318, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 9.529228
Epoch 15
Validation binary_cross_entropy = 6.640479
Epoch 16
Validation binary_cross_entropy = 20.249805
Epoch 17
Validation binary_cross_entropy = 33.094769
Epoch 18
Validation binary_cross_entropy = 6.250099
Epoch 19
Loss = 3.5647e+01, PNorm = 83.9210, GNorm = 668.8274, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 25.017532
Epoch 20
Validation binary_cross_entropy = 38.356487
Epoch 21
Validation binary_cross_entropy = 6.670613
Epoch 22
Validation binary_cross_entropy = 6.387021
Epoch 23
Validation binary_cross_entropy = 76.184967
Epoch 24
Loss = 5.4171e+01, PNorm = 84.1263, GNorm = 1186.8932, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 62.458538
Epoch 25
Validation binary_cross_entropy = 6.895848
Epoch 26
Validation binary_cross_entropy = 7.769583
Epoch 27
Validation binary_cross_entropy = 30.396475
Epoch 28
Validation binary_cross_entropy = 9.890340
Epoch 29
Loss = 1.9406e+01, PNorm = 84.2688, GNorm = 595.9552, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 6.372240
Epoch 30
Validation binary_cross_entropy = 6.326418
Epoch 31
Validation binary_cross_entropy = 19.265629
Epoch 32
Validation binary_cross_entropy = 9.099119
Epoch 33
Validation binary_cross_entropy = 5.634343
Epoch 34
Loss = 1.3121e+01, PNorm = 84.3794, GNorm = 498.7104, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 5.547609
Epoch 35
Validation binary_cross_entropy = 6.107871
Epoch 36
Validation binary_cross_entropy = 5.941965
Epoch 37
Validation binary_cross_entropy = 5.438044
Epoch 38
Validation binary_cross_entropy = 5.543349
Epoch 39
Loss = 1.0938e+01, PNorm = 84.4792, GNorm = 196.7822, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 6.679352
Epoch 40
Validation binary_cross_entropy = 7.141318
Epoch 41
Validation binary_cross_entropy = 6.500241
Epoch 42
Validation binary_cross_entropy = 5.618721
Epoch 43
Validation binary_cross_entropy = 6.365411
Epoch 44
Loss = 1.1687e+01, PNorm = 84.6187, GNorm = 431.4325, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 7.024386
Epoch 45
Validation binary_cross_entropy = 7.759613
Epoch 46
Validation binary_cross_entropy = 4.971119
Epoch 47
Validation binary_cross_entropy = 4.779188
Epoch 48
Validation binary_cross_entropy = 4.371567
Epoch 49
Loss = 1.0628e+01, PNorm = 84.8028, GNorm = 258.8678, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 4.222096
Epoch 50
Validation binary_cross_entropy = 5.503600
Epoch 51
Validation binary_cross_entropy = 4.404946
Epoch 52
Validation binary_cross_entropy = 7.624144
Epoch 53
Validation binary_cross_entropy = 3.834815
Epoch 54
Loss = 8.7545e+00, PNorm = 85.1222, GNorm = 177.3099, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 4.399730
Epoch 55
Validation binary_cross_entropy = 3.571809
Epoch 56
Validation binary_cross_entropy = 2.372866
Epoch 57
Validation binary_cross_entropy = 2.196476
Epoch 58
Validation binary_cross_entropy = 3.976927
Epoch 59
Loss = 6.3231e+00, PNorm = 85.3691, GNorm = 232.5477, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 2.917909
Epoch 60
Validation binary_cross_entropy = 1.979329
Epoch 61
Validation binary_cross_entropy = 1.739383
Epoch 62
Validation binary_cross_entropy = 2.152169
Epoch 63
Validation binary_cross_entropy = 2.150159
Epoch 64
Loss = 1.8323e+00, PNorm = 85.5672, GNorm = 59.6906, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.679590
Epoch 65
Validation binary_cross_entropy = 2.099809
Epoch 66
Validation binary_cross_entropy = 1.625543
Epoch 67
Validation binary_cross_entropy = 2.127241
Epoch 68
Validation binary_cross_entropy = 1.801937
Epoch 69
Loss = 1.7160e+00, PNorm = 85.7655, GNorm = 110.8037, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 3.227175
Epoch 70
Validation binary_cross_entropy = 5.819688
Epoch 71
Validation binary_cross_entropy = 1.624537
Epoch 72
Validation binary_cross_entropy = 1.651243
Epoch 73
Validation binary_cross_entropy = 1.482796
Epoch 74
Loss = 3.6878e+00, PNorm = 85.8907, GNorm = 125.6830, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.034972
Epoch 75
Validation binary_cross_entropy = 3.208410
Epoch 76
Validation binary_cross_entropy = 1.953768
Epoch 77
Validation binary_cross_entropy = 1.455086
Epoch 78
Validation binary_cross_entropy = 1.966236
Epoch 79
Loss = 2.3134e+00, PNorm = 86.0413, GNorm = 84.4579, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.095592
Epoch 80
Validation binary_cross_entropy = 2.005407
Epoch 81
Validation binary_cross_entropy = 2.500753
Epoch 82
Validation binary_cross_entropy = 1.877501
Epoch 83
Validation binary_cross_entropy = 1.298434
Epoch 84
Loss = 2.6736e+00, PNorm = 86.1206, GNorm = 126.8470, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.236488
Epoch 85
Validation binary_cross_entropy = 1.632498
Epoch 86
Validation binary_cross_entropy = 1.875322
Epoch 87
Validation binary_cross_entropy = 1.029426
Epoch 88
Validation binary_cross_entropy = 1.030954
Epoch 89
Loss = 1.9043e+00, PNorm = 86.2047, GNorm = 139.4069, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.035024
Epoch 90
Validation binary_cross_entropy = 1.178757
Epoch 91
Validation binary_cross_entropy = 2.585338
Epoch 92
Validation binary_cross_entropy = 2.475120
Epoch 93
Validation binary_cross_entropy = 2.456289
Epoch 94
Loss = 2.7045e-01, PNorm = 86.3007, GNorm = 45.4786, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.357927
Epoch 95
Validation binary_cross_entropy = 1.241778
Epoch 96
Validation binary_cross_entropy = 1.140499
Epoch 97
Validation binary_cross_entropy = 1.139633
Epoch 98
Validation binary_cross_entropy = 1.156889
Epoch 99
Loss = 1.9634e+00, PNorm = 86.3839, GNorm = 59.0074, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.187993
Epoch 100
Validation binary_cross_entropy = 1.212098
Epoch 101
Validation binary_cross_entropy = 1.185707
Epoch 102
Validation binary_cross_entropy = 1.062700
Epoch 103
Validation binary_cross_entropy = 1.040335
Epoch 104
Loss = 8.4832e-01, PNorm = 86.4821, GNorm = 102.4279, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.928094
Epoch 105
Validation binary_cross_entropy = 0.932339
Epoch 106
Validation binary_cross_entropy = 1.025486
Epoch 107
Validation binary_cross_entropy = 3.195467
Epoch 108
Validation binary_cross_entropy = 2.178605
Epoch 109
Loss = 1.7246e+00, PNorm = 86.5775, GNorm = 66.4938, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.812060
Epoch 110
Validation binary_cross_entropy = 0.799513
Epoch 111
Validation binary_cross_entropy = 0.879838
Epoch 112
Validation binary_cross_entropy = 0.932701
Epoch 113
Validation binary_cross_entropy = 1.011842
Epoch 114
Loss = 1.3225e+00, PNorm = 86.6796, GNorm = 108.0417, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.026263
Epoch 115
Validation binary_cross_entropy = 0.926928
Epoch 116
Validation binary_cross_entropy = 0.774931
Epoch 117
Validation binary_cross_entropy = 0.739086
Epoch 118
Validation binary_cross_entropy = 0.742278
Epoch 119
Loss = 1.3455e+00, PNorm = 86.7869, GNorm = 39.9623, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.769865
Epoch 120
Validation binary_cross_entropy = 0.803715
Epoch 121
Validation binary_cross_entropy = 0.887319
Epoch 122
Validation binary_cross_entropy = 0.777738
Epoch 123
Validation binary_cross_entropy = 0.702153
Epoch 124
Loss = 1.2396e+00, PNorm = 86.9278, GNorm = 62.2154, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.707263
Epoch 125
Validation binary_cross_entropy = 0.658206
Epoch 126
Validation binary_cross_entropy = 0.584201
Epoch 127
Validation binary_cross_entropy = 0.572646
Epoch 128
Validation binary_cross_entropy = 0.609188
Epoch 129
Loss = 9.6134e-01, PNorm = 87.0708, GNorm = 69.4518, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.673968
Epoch 130
Validation binary_cross_entropy = 0.599132
Epoch 131
Validation binary_cross_entropy = 0.517142
Epoch 132
Validation binary_cross_entropy = 0.520831
Epoch 133
Validation binary_cross_entropy = 0.521478
Epoch 134
Loss = 1.0187e+00, PNorm = 87.1752, GNorm = 56.1736, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.511921
Epoch 135
Validation binary_cross_entropy = 0.527133
Epoch 136
Validation binary_cross_entropy = 0.552059
Epoch 137
Validation binary_cross_entropy = 0.601840
Epoch 138
Validation binary_cross_entropy = 0.611897
Epoch 139
Loss = 8.8689e-01, PNorm = 87.3014, GNorm = 55.3759, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.612209
Epoch 140
Validation binary_cross_entropy = 0.535455
Epoch 141
Validation binary_cross_entropy = 0.537719
Epoch 142
Validation binary_cross_entropy = 0.533663
Epoch 143
Validation binary_cross_entropy = 0.682622
Epoch 144
Loss = 1.2433e+00, PNorm = 87.4326, GNorm = 55.1045, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.576829
Epoch 145
Validation binary_cross_entropy = 0.839078
Epoch 146
Validation binary_cross_entropy = 0.476947
Epoch 147
Validation binary_cross_entropy = 0.525994
Epoch 148
Validation binary_cross_entropy = 0.475294
Epoch 149
Loss = 8.6443e-01, PNorm = 87.5266, GNorm = 50.9320, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.600929
Epoch 150
Validation binary_cross_entropy = 1.060823
Epoch 151
Validation binary_cross_entropy = 1.302660
Epoch 152
Validation binary_cross_entropy = 0.746344
Epoch 153
Validation binary_cross_entropy = 0.487300
Epoch 154
Loss = 1.2381e+00, PNorm = 87.6401, GNorm = 102.3893, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.472304
Epoch 155
Validation binary_cross_entropy = 0.525023
Epoch 156
Validation binary_cross_entropy = 0.632021
Epoch 157
Validation binary_cross_entropy = 1.332126
Epoch 158
Validation binary_cross_entropy = 1.441085
Epoch 159
Loss = 1.2898e+00, PNorm = 87.7553, GNorm = 36.6814, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.321976
Epoch 160
Validation binary_cross_entropy = 0.655060
Epoch 161
Validation binary_cross_entropy = 0.573436
Epoch 162
Validation binary_cross_entropy = 0.552120
Epoch 163
Validation binary_cross_entropy = 0.551626
Epoch 164
Loss = 1.0745e+00, PNorm = 87.8395, GNorm = 58.4511, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.539151
Epoch 165
Validation binary_cross_entropy = 0.540048
Epoch 166
Validation binary_cross_entropy = 0.524754
Epoch 167
Validation binary_cross_entropy = 0.487489
Epoch 168
Validation binary_cross_entropy = 0.483757
Epoch 169
Loss = 6.0155e-01, PNorm = 87.9058, GNorm = 58.1589, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.479818
Epoch 170
Validation binary_cross_entropy = 0.514243
Epoch 171
Validation binary_cross_entropy = 0.526582
Epoch 172
Validation binary_cross_entropy = 0.476656
Epoch 173
Validation binary_cross_entropy = 0.446799
Epoch 174
Loss = 7.5553e-01, PNorm = 88.0030, GNorm = 36.4118, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.416532
Epoch 175
Validation binary_cross_entropy = 0.435100
Epoch 176
Validation binary_cross_entropy = 0.460719
Epoch 177
Validation binary_cross_entropy = 0.487245
Epoch 178
Validation binary_cross_entropy = 0.530037
Epoch 179
Loss = 3.5177e-01, PNorm = 88.1237, GNorm = 24.6935, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.541070
Epoch 180
Validation binary_cross_entropy = 0.501857
Epoch 181
Validation binary_cross_entropy = 0.474957
Epoch 182
Validation binary_cross_entropy = 0.492762
Epoch 183
Validation binary_cross_entropy = 0.549538
Epoch 184
Loss = 9.1094e-01, PNorm = 88.2191, GNorm = 53.8643, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.582966
Epoch 185
Validation binary_cross_entropy = 0.623071
Epoch 186
Validation binary_cross_entropy = 0.635356
Epoch 187
Validation binary_cross_entropy = 0.602194
Epoch 188
Validation binary_cross_entropy = 0.602564
Epoch 189
Loss = 5.1858e-01, PNorm = 88.3090, GNorm = 21.8924, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.574173
Epoch 190
Validation binary_cross_entropy = 0.520500
Epoch 191
Validation binary_cross_entropy = 0.592230
Epoch 192
Validation binary_cross_entropy = 0.732932
Epoch 193
Validation binary_cross_entropy = 0.847624
Epoch 194
Loss = 7.9518e-01, PNorm = 88.4276, GNorm = 28.8146, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.834820
Epoch 195
Validation binary_cross_entropy = 0.665541
Epoch 196
Validation binary_cross_entropy = 0.590974
Epoch 197
Validation binary_cross_entropy = 0.584082
Epoch 198
Validation binary_cross_entropy = 0.593024
Epoch 199
Loss = 1.9499e-01, PNorm = 88.5184, GNorm = 26.8011, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.598109
Epoch 200
Validation binary_cross_entropy = 0.686079
Epoch 201
Validation binary_cross_entropy = 0.743774
Epoch 202
Validation binary_cross_entropy = 0.567329
Epoch 203
Validation binary_cross_entropy = 0.522179
Epoch 204
Loss = 1.1257e-01, PNorm = 88.6287, GNorm = 18.7987, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.512708
Epoch 205
Validation binary_cross_entropy = 0.538359
Epoch 206
Validation binary_cross_entropy = 0.555880
Epoch 207
Validation binary_cross_entropy = 0.489103
Epoch 208
Validation binary_cross_entropy = 0.428859
Epoch 209
Loss = 6.1252e-01, PNorm = 88.7400, GNorm = 58.2022, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.420140
Epoch 210
Validation binary_cross_entropy = 0.414886
Epoch 211
Validation binary_cross_entropy = 0.425218
Epoch 212
Validation binary_cross_entropy = 0.464110
Epoch 213
Validation binary_cross_entropy = 0.496536
Epoch 214
Loss = 4.6275e-01, PNorm = 88.8469, GNorm = 17.7870, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.456907
Epoch 215
Validation binary_cross_entropy = 0.419587
Epoch 216
Validation binary_cross_entropy = 0.420940
Epoch 217
Validation binary_cross_entropy = 0.419145
Epoch 218
Validation binary_cross_entropy = 0.427379
Epoch 219
Loss = 3.2141e-01, PNorm = 88.9355, GNorm = 12.8631, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.450281
Epoch 220
Validation binary_cross_entropy = 0.477532
Epoch 221
Validation binary_cross_entropy = 0.475175
Epoch 222
Validation binary_cross_entropy = 0.438105
Epoch 223
Validation binary_cross_entropy = 0.410189
Epoch 224
Loss = 2.1564e-01, PNorm = 88.9984, GNorm = 18.8088, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.394484
Epoch 225
Validation binary_cross_entropy = 0.387186
Epoch 226
Validation binary_cross_entropy = 0.380855
Epoch 227
Validation binary_cross_entropy = 0.391652
Epoch 228
Validation binary_cross_entropy = 0.388642
Epoch 229
Loss = 4.3669e-01, PNorm = 89.0807, GNorm = 26.5137, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.417188
Epoch 230
Validation binary_cross_entropy = 0.461911
Epoch 231
Validation binary_cross_entropy = 0.446455
Epoch 232
Validation binary_cross_entropy = 0.388431
Epoch 233
Validation binary_cross_entropy = 0.399031
Epoch 234
Loss = 1.7730e-01, PNorm = 89.1734, GNorm = 34.6679, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.448397
Epoch 235
Validation binary_cross_entropy = 0.548254
Epoch 236
Validation binary_cross_entropy = 0.633004
Epoch 237
Validation binary_cross_entropy = 0.527625
Epoch 238
Validation binary_cross_entropy = 0.417632
Epoch 239
Loss = 1.5417e-01, PNorm = 89.2531, GNorm = 20.4043, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.382095
Epoch 240
Validation binary_cross_entropy = 0.379203
Epoch 241
Validation binary_cross_entropy = 0.427330
Epoch 242
Validation binary_cross_entropy = 0.492221
Epoch 243
Validation binary_cross_entropy = 0.552927
Epoch 244
Loss = 1.1888e-01, PNorm = 89.3851, GNorm = 3.8009, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.538385
Epoch 245
Validation binary_cross_entropy = 0.495131
Epoch 246
Validation binary_cross_entropy = 0.419607
Epoch 247
Validation binary_cross_entropy = 0.405768
Epoch 248
Validation binary_cross_entropy = 0.405387
Epoch 249
Loss = 5.7560e-01, PNorm = 89.4923, GNorm = 33.1977, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.412482
Epoch 250
Validation binary_cross_entropy = 0.449717
Epoch 251
Validation binary_cross_entropy = 0.506804
Epoch 252
Validation binary_cross_entropy = 0.535968
Epoch 253
Validation binary_cross_entropy = 0.494571
Epoch 254
Loss = 2.2341e-01, PNorm = 89.5642, GNorm = 32.9649, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.470068
Epoch 255
Validation binary_cross_entropy = 0.442689
Epoch 256
Validation binary_cross_entropy = 0.416501
Epoch 257
Validation binary_cross_entropy = 0.417315
Epoch 258
Validation binary_cross_entropy = 0.453182
Epoch 259
Loss = 5.5612e-01, PNorm = 89.6554, GNorm = 41.3255, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.506440
Epoch 260
Validation binary_cross_entropy = 0.540409
Epoch 261
Validation binary_cross_entropy = 0.548404
Epoch 262
Validation binary_cross_entropy = 0.497632
Epoch 263
Validation binary_cross_entropy = 0.442602
Epoch 264
Loss = 2.8321e-01, PNorm = 89.7680, GNorm = 17.1530, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.414240
Epoch 265
Validation binary_cross_entropy = 0.397922
Epoch 266
Validation binary_cross_entropy = 0.424802
Epoch 267
Validation binary_cross_entropy = 0.491143
Epoch 268
Validation binary_cross_entropy = 0.505140
Epoch 269
Loss = 4.1771e-01, PNorm = 89.8537, GNorm = 36.6285, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.450524
Epoch 270
Validation binary_cross_entropy = 0.383830
Epoch 271
Validation binary_cross_entropy = 0.366559
Epoch 272
Validation binary_cross_entropy = 0.365158
Epoch 273
Validation binary_cross_entropy = 0.364183
Epoch 274
Loss = 2.7214e-01, PNorm = 89.9289, GNorm = 11.6226, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.374218
Epoch 275
Validation binary_cross_entropy = 0.409346
Epoch 276
Validation binary_cross_entropy = 0.450782
Epoch 277
Validation binary_cross_entropy = 0.456182
Epoch 278
Validation binary_cross_entropy = 0.451546
Epoch 279
Loss = 2.7759e-01, PNorm = 90.0172, GNorm = 18.6378, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.435034
Epoch 280
Validation binary_cross_entropy = 0.396871
Epoch 281
Validation binary_cross_entropy = 0.363382
Epoch 282
Validation binary_cross_entropy = 0.355003
Epoch 283
Validation binary_cross_entropy = 0.358611
Epoch 284
Loss = 4.2321e-01, PNorm = 90.1077, GNorm = 43.6200, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.366570
Epoch 285
Validation binary_cross_entropy = 0.374613
Epoch 286
Validation binary_cross_entropy = 0.396957
Epoch 287
Validation binary_cross_entropy = 0.407233
Epoch 288
Validation binary_cross_entropy = 0.393234
Epoch 289
Loss = 2.4010e-01, PNorm = 90.1913, GNorm = 30.5514, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.376619
Epoch 290
Validation binary_cross_entropy = 0.365959
Epoch 291
Validation binary_cross_entropy = 0.358612
Epoch 292
Validation binary_cross_entropy = 0.351495
Epoch 293
Validation binary_cross_entropy = 0.339831
Epoch 294
Loss = 3.5281e-01, PNorm = 90.2827, GNorm = 28.9967, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.343515
Epoch 295
Validation binary_cross_entropy = 0.359105
Epoch 296
Validation binary_cross_entropy = 0.357642
Epoch 297
Validation binary_cross_entropy = 0.350544
Epoch 298
Validation binary_cross_entropy = 0.344230
Epoch 299
Loss = 1.8846e-01, PNorm = 90.3963, GNorm = 17.4099, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.349641
Model 0 best validation binary_cross_entropy = 0.339831 on epoch 293
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.156751
Ensemble test binary_cross_entropy = 0.156751
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 5.444703
Epoch 1
Validation binary_cross_entropy = 4.760542
Epoch 2
Validation binary_cross_entropy = 25.152378
Epoch 3
Validation binary_cross_entropy = 6.334114
Epoch 4
Loss = 5.2131e+01, PNorm = 82.9658, GNorm = 1138.2336, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 12.103530
Epoch 5
Validation binary_cross_entropy = 6.533638
Epoch 6
Validation binary_cross_entropy = 20.334522
Epoch 7
Validation binary_cross_entropy = 7.093575
Epoch 8
Validation binary_cross_entropy = 11.132923
Epoch 9
Loss = 3.0594e+01, PNorm = 83.2578, GNorm = 605.1426, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 31.837175
Epoch 10
Validation binary_cross_entropy = 7.019069
Epoch 11
Validation binary_cross_entropy = 63.542191
Epoch 12
Validation binary_cross_entropy = 11.164032
Epoch 13
Validation binary_cross_entropy = 6.211452
Epoch 14
Loss = 2.7509e+01, PNorm = 83.5879, GNorm = 500.8912, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 37.044880
Epoch 15
Validation binary_cross_entropy = 7.997349
Epoch 16
Validation binary_cross_entropy = 5.623125
Epoch 17
Validation binary_cross_entropy = 34.263058
Epoch 18
Validation binary_cross_entropy = 5.687884
Epoch 19
Loss = 1.9269e+01, PNorm = 83.8499, GNorm = 507.4442, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 5.968207
Epoch 20
Validation binary_cross_entropy = 11.795998
Epoch 21
Validation binary_cross_entropy = 13.527843
Epoch 22
Validation binary_cross_entropy = 6.675364
Epoch 23
Validation binary_cross_entropy = 5.810693
Epoch 24
Loss = 2.0861e+01, PNorm = 84.0323, GNorm = 393.1481, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 15.691682
Epoch 25
Validation binary_cross_entropy = 42.186539
Epoch 26
Validation binary_cross_entropy = 4.756016
Epoch 27
Validation binary_cross_entropy = 5.925757
Epoch 28
Validation binary_cross_entropy = 9.655204
Epoch 29
Loss = 1.1009e+01, PNorm = 84.1676, GNorm = 545.6807, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 53.020706
Epoch 30
Validation binary_cross_entropy = 11.508398
Epoch 31
Validation binary_cross_entropy = 4.142996
Epoch 32
Validation binary_cross_entropy = 4.926561
Epoch 33
Validation binary_cross_entropy = 31.722305
Epoch 34
Loss = 1.0260e+01, PNorm = 84.2820, GNorm = 312.9515, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 37.299015
Epoch 35
Validation binary_cross_entropy = 15.563825
Epoch 36
Validation binary_cross_entropy = 5.698792
Epoch 37
Validation binary_cross_entropy = 5.075205
Epoch 38
Validation binary_cross_entropy = 5.241441
Epoch 39
Loss = 1.2034e+01, PNorm = 84.4114, GNorm = 260.3555, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 7.729660
Epoch 40
Validation binary_cross_entropy = 5.732694
Epoch 41
Validation binary_cross_entropy = 4.152667
Epoch 42
Validation binary_cross_entropy = 4.533060
Epoch 43
Validation binary_cross_entropy = 5.856424
Epoch 44
Loss = 5.2754e+00, PNorm = 84.5264, GNorm = 194.4392, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 6.403537
Epoch 45
Validation binary_cross_entropy = 3.656491
Epoch 46
Validation binary_cross_entropy = 3.283378
Epoch 47
Validation binary_cross_entropy = 4.443731
Epoch 48
Validation binary_cross_entropy = 17.496841
Epoch 49
Loss = 8.4433e+00, PNorm = 84.6610, GNorm = 352.0577, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 9.372474
Epoch 50
Validation binary_cross_entropy = 3.650776
Epoch 51
Validation binary_cross_entropy = 2.162474
Epoch 52
Validation binary_cross_entropy = 3.083485
Epoch 53
Validation binary_cross_entropy = 5.339569
Epoch 54
Loss = 5.9852e+00, PNorm = 84.8161, GNorm = 203.5935, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 12.834832
Epoch 55
Validation binary_cross_entropy = 4.446321
Epoch 56
Validation binary_cross_entropy = 2.821018
Epoch 57
Validation binary_cross_entropy = 3.381898
Epoch 58
Validation binary_cross_entropy = 3.385373
Epoch 59
Loss = 3.0843e+00, PNorm = 84.9811, GNorm = 112.8419, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 5.446092
Epoch 60
Validation binary_cross_entropy = 5.994080
Epoch 61
Validation binary_cross_entropy = 4.830816
Epoch 62
Validation binary_cross_entropy = 2.699460
Epoch 63
Validation binary_cross_entropy = 2.615491
Epoch 64
Loss = 2.9738e+00, PNorm = 85.1265, GNorm = 98.5523, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 2.551458
Epoch 65
Validation binary_cross_entropy = 3.407248
Epoch 66
Validation binary_cross_entropy = 9.906252
Epoch 67
Validation binary_cross_entropy = 10.562860
Epoch 68
Validation binary_cross_entropy = 4.982703
Epoch 69
Loss = 1.8046e+00, PNorm = 85.2309, GNorm = 147.5619, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.538529
Epoch 70
Validation binary_cross_entropy = 1.370974
Epoch 71
Validation binary_cross_entropy = 2.125870
Epoch 72
Validation binary_cross_entropy = 2.127763
Epoch 73
Validation binary_cross_entropy = 1.426240
Epoch 74
Loss = 2.3202e+00, PNorm = 85.3442, GNorm = 109.7476, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.568175
Epoch 75
Validation binary_cross_entropy = 1.451738
Epoch 76
Validation binary_cross_entropy = 0.969121
Epoch 77
Validation binary_cross_entropy = 0.986119
Epoch 78
Validation binary_cross_entropy = 2.606370
Epoch 79
Loss = 2.5976e+00, PNorm = 85.5269, GNorm = 118.5977, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 3.031029
Epoch 80
Validation binary_cross_entropy = 2.220697
Epoch 81
Validation binary_cross_entropy = 1.488252
Epoch 82
Validation binary_cross_entropy = 0.945893
Epoch 83
Validation binary_cross_entropy = 0.904563
Epoch 84
Loss = 2.4785e+00, PNorm = 85.6468, GNorm = 110.4014, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.814491
Epoch 85
Validation binary_cross_entropy = 0.767175
Epoch 86
Validation binary_cross_entropy = 0.702847
Epoch 87
Validation binary_cross_entropy = 0.770698
Epoch 88
Validation binary_cross_entropy = 0.745086
Epoch 89
Loss = 2.2804e+00, PNorm = 85.8549, GNorm = 119.3944, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.641429
Epoch 90
Validation binary_cross_entropy = 0.719943
Epoch 91
Validation binary_cross_entropy = 0.655202
Epoch 92
Validation binary_cross_entropy = 0.908998
Epoch 93
Validation binary_cross_entropy = 1.462489
Epoch 94
Loss = 3.1078e+00, PNorm = 86.0052, GNorm = 131.9308, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.586461
Epoch 95
Validation binary_cross_entropy = 0.666573
Epoch 96
Validation binary_cross_entropy = 0.677831
Epoch 97
Validation binary_cross_entropy = 0.530790
Epoch 98
Validation binary_cross_entropy = 0.674821
Epoch 99
Loss = 7.6516e-01, PNorm = 86.1740, GNorm = 71.5431, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.935091
Epoch 100
Validation binary_cross_entropy = 0.579252
Epoch 101
Validation binary_cross_entropy = 0.549044
Epoch 102
Validation binary_cross_entropy = 0.624241
Epoch 103
Validation binary_cross_entropy = 0.642677
Epoch 104
Loss = 1.2044e+00, PNorm = 86.2926, GNorm = 93.5281, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.582912
Epoch 105
Validation binary_cross_entropy = 0.594789
Epoch 106
Validation binary_cross_entropy = 0.620894
Epoch 107
Validation binary_cross_entropy = 0.596336
Epoch 108
Validation binary_cross_entropy = 0.679805
Epoch 109
Loss = 1.1321e+00, PNorm = 86.4176, GNorm = 56.0954, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.694128
Epoch 110
Validation binary_cross_entropy = 0.529430
Epoch 111
Validation binary_cross_entropy = 0.485967
Epoch 112
Validation binary_cross_entropy = 0.506365
Epoch 113
Validation binary_cross_entropy = 0.543266
Epoch 114
Loss = 1.0688e+00, PNorm = 86.5326, GNorm = 93.2367, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.536598
Epoch 115
Validation binary_cross_entropy = 0.510159
Epoch 116
Validation binary_cross_entropy = 0.505116
Epoch 117
Validation binary_cross_entropy = 0.542660
Epoch 118
Validation binary_cross_entropy = 0.515478
Epoch 119
Loss = 8.4885e-01, PNorm = 86.6543, GNorm = 47.4400, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.521830
Epoch 120
Validation binary_cross_entropy = 0.522334
Epoch 121
Validation binary_cross_entropy = 0.511002
Epoch 122
Validation binary_cross_entropy = 0.511887
Epoch 123
Validation binary_cross_entropy = 0.543868
Epoch 124
Loss = 1.2702e+00, PNorm = 86.7757, GNorm = 89.0742, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.508724
Epoch 125
Validation binary_cross_entropy = 0.501662
Epoch 126
Validation binary_cross_entropy = 0.511894
Epoch 127
Validation binary_cross_entropy = 0.554417
Epoch 128
Validation binary_cross_entropy = 0.624946
Epoch 129
Loss = 4.1097e-01, PNorm = 86.9226, GNorm = 29.8206, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.712690
Epoch 130
Validation binary_cross_entropy = 0.562789
Epoch 131
Validation binary_cross_entropy = 0.515873
Epoch 132
Validation binary_cross_entropy = 0.522207
Epoch 133
Validation binary_cross_entropy = 0.556293
Epoch 134
Loss = 4.8165e-01, PNorm = 87.0762, GNorm = 43.6932, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.606063
Epoch 135
Validation binary_cross_entropy = 0.705768
Epoch 136
Validation binary_cross_entropy = 0.867459
Epoch 137
Validation binary_cross_entropy = 0.653601
Epoch 138
Validation binary_cross_entropy = 0.549082
Epoch 139
Loss = 7.4674e-01, PNorm = 87.2376, GNorm = 54.8099, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.545652
Epoch 140
Validation binary_cross_entropy = 0.612737
Epoch 141
Validation binary_cross_entropy = 0.668020
Epoch 142
Validation binary_cross_entropy = 0.674610
Epoch 143
Validation binary_cross_entropy = 0.640655
Epoch 144
Loss = 5.5362e-01, PNorm = 87.4132, GNorm = 9.4396, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.544028
Epoch 145
Validation binary_cross_entropy = 0.509608
Epoch 146
Validation binary_cross_entropy = 0.543220
Epoch 147
Validation binary_cross_entropy = 0.654660
Epoch 148
Validation binary_cross_entropy = 0.800333
Epoch 149
Loss = 1.5940e+00, PNorm = 87.5540, GNorm = 85.7141, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.777869
Epoch 150
Validation binary_cross_entropy = 0.673108
Epoch 151
Validation binary_cross_entropy = 0.589343
Epoch 152
Validation binary_cross_entropy = 0.510565
Epoch 153
Validation binary_cross_entropy = 0.492462
Epoch 154
Loss = 6.4937e-01, PNorm = 87.6674, GNorm = 46.1484, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.514515
Epoch 155
Validation binary_cross_entropy = 0.537121
Epoch 156
Validation binary_cross_entropy = 0.509167
Epoch 157
Validation binary_cross_entropy = 0.452675
Epoch 158
Validation binary_cross_entropy = 0.441196
Epoch 159
Loss = 5.4440e-01, PNorm = 87.7721, GNorm = 46.0869, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.458831
Epoch 160
Validation binary_cross_entropy = 0.475265
Epoch 161
Validation binary_cross_entropy = 0.493051
Epoch 162
Validation binary_cross_entropy = 0.476641
Epoch 163
Validation binary_cross_entropy = 0.485541
Epoch 164
Loss = 8.5253e-01, PNorm = 87.8738, GNorm = 53.0787, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.484854
Epoch 165
Validation binary_cross_entropy = 0.446253
Epoch 166
Validation binary_cross_entropy = 0.436766
Epoch 167
Validation binary_cross_entropy = 0.419033
Epoch 168
Validation binary_cross_entropy = 0.412005
Epoch 169
Loss = 6.1729e-01, PNorm = 87.9508, GNorm = 30.9689, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.408480
Epoch 170
Validation binary_cross_entropy = 0.409566
Epoch 171
Validation binary_cross_entropy = 0.410264
Epoch 172
Validation binary_cross_entropy = 0.430497
Epoch 173
Validation binary_cross_entropy = 0.505026
Epoch 174
Loss = 4.6801e-01, PNorm = 88.0600, GNorm = 25.0966, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.561803
Epoch 175
Validation binary_cross_entropy = 0.570097
Epoch 176
Validation binary_cross_entropy = 0.509048
Epoch 177
Validation binary_cross_entropy = 0.515175
Epoch 178
Validation binary_cross_entropy = 0.513747
Epoch 179
Loss = 4.8867e-01, PNorm = 88.1945, GNorm = 16.7909, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.496449
Epoch 180
Validation binary_cross_entropy = 0.436306
Epoch 181
Validation binary_cross_entropy = 0.414047
Epoch 182
Validation binary_cross_entropy = 0.411213
Epoch 183
Validation binary_cross_entropy = 0.444735
Epoch 184
Loss = 3.1309e-01, PNorm = 88.2971, GNorm = 29.8552, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.536645
Epoch 185
Validation binary_cross_entropy = 0.647732
Epoch 186
Validation binary_cross_entropy = 0.561749
Epoch 187
Validation binary_cross_entropy = 0.483924
Epoch 188
Validation binary_cross_entropy = 0.421250
Epoch 189
Loss = 7.0811e-01, PNorm = 88.3478, GNorm = 26.0435, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.407655
Epoch 190
Validation binary_cross_entropy = 0.416203
Epoch 191
Validation binary_cross_entropy = 0.441687
Epoch 192
Validation binary_cross_entropy = 0.471169
Epoch 193
Validation binary_cross_entropy = 0.613092
Epoch 194
Loss = 5.3825e-01, PNorm = 88.4540, GNorm = 44.7001, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.590353
Epoch 195
Validation binary_cross_entropy = 0.512633
Epoch 196
Validation binary_cross_entropy = 0.436077
Epoch 197
Validation binary_cross_entropy = 0.407196
Epoch 198
Validation binary_cross_entropy = 0.398432
Epoch 199
Loss = 6.3826e-01, PNorm = 88.5715, GNorm = 48.4010, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.404988
Epoch 200
Validation binary_cross_entropy = 0.448296
Epoch 201
Validation binary_cross_entropy = 0.499662
Epoch 202
Validation binary_cross_entropy = 0.497092
Epoch 203
Validation binary_cross_entropy = 0.460004
Epoch 204
Loss = 5.0457e-01, PNorm = 88.6722, GNorm = 51.7496, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.411081
Epoch 205
Validation binary_cross_entropy = 0.381627
Epoch 206
Validation binary_cross_entropy = 0.368944
Epoch 207
Validation binary_cross_entropy = 0.365751
Epoch 208
Validation binary_cross_entropy = 0.375794
Epoch 209
Loss = 7.7442e-01, PNorm = 88.7444, GNorm = 65.1890, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.399766
Epoch 210
Validation binary_cross_entropy = 0.405857
Epoch 211
Validation binary_cross_entropy = 0.393637
Epoch 212
Validation binary_cross_entropy = 0.389323
Epoch 213
Validation binary_cross_entropy = 0.414761
Epoch 214
Loss = 4.8384e-01, PNorm = 88.8641, GNorm = 33.8614, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.405076
Epoch 215
Validation binary_cross_entropy = 0.378771
Epoch 216
Validation binary_cross_entropy = 0.375587
Epoch 217
Validation binary_cross_entropy = 0.404647
Epoch 218
Validation binary_cross_entropy = 0.427599
Epoch 219
Loss = 5.0282e-01, PNorm = 88.9801, GNorm = 23.8359, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.463236
Epoch 220
Validation binary_cross_entropy = 0.435225
Epoch 221
Validation binary_cross_entropy = 0.412828
Epoch 222
Validation binary_cross_entropy = 0.392451
Epoch 223
Validation binary_cross_entropy = 0.379290
Epoch 224
Loss = 6.4104e-01, PNorm = 89.0558, GNorm = 38.2240, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.384888
Epoch 225
Validation binary_cross_entropy = 0.397686
Epoch 226
Validation binary_cross_entropy = 0.445126
Epoch 227
Validation binary_cross_entropy = 0.490923
Epoch 228
Validation binary_cross_entropy = 0.473584
Epoch 229
Loss = 6.0654e-01, PNorm = 89.1396, GNorm = 38.9232, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.417176
Epoch 230
Validation binary_cross_entropy = 0.377056
Epoch 231
Validation binary_cross_entropy = 0.381043
Epoch 232
Validation binary_cross_entropy = 0.398427
Epoch 233
Validation binary_cross_entropy = 0.433162
Epoch 234
Loss = 4.5608e-01, PNorm = 89.2475, GNorm = 23.9614, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.489887
Epoch 235
Validation binary_cross_entropy = 0.508938
Epoch 236
Validation binary_cross_entropy = 0.469834
Epoch 237
Validation binary_cross_entropy = 0.419332
Epoch 238
Validation binary_cross_entropy = 0.425488
Epoch 239
Loss = 4.8622e-01, PNorm = 89.3480, GNorm = 27.0756, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.435190
Epoch 240
Validation binary_cross_entropy = 0.452719
Epoch 241
Validation binary_cross_entropy = 0.497323
Epoch 242
Validation binary_cross_entropy = 0.564020
Epoch 243
Validation binary_cross_entropy = 0.571010
Epoch 244
Loss = 4.4354e-01, PNorm = 89.4390, GNorm = 28.6551, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.474068
Epoch 245
Validation binary_cross_entropy = 0.393991
Epoch 246
Validation binary_cross_entropy = 0.380776
Epoch 247
Validation binary_cross_entropy = 0.367480
Epoch 248
Validation binary_cross_entropy = 0.372079
Epoch 249
Loss = 5.4554e-01, PNorm = 89.5088, GNorm = 68.7733, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.394589
Epoch 250
Validation binary_cross_entropy = 0.472333
Epoch 251
Validation binary_cross_entropy = 0.533763
Epoch 252
Validation binary_cross_entropy = 0.530982
Epoch 253
Validation binary_cross_entropy = 0.441250
Epoch 254
Loss = 3.5045e-01, PNorm = 89.6067, GNorm = 29.2602, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.381659
Epoch 255
Validation binary_cross_entropy = 0.349148
Epoch 256
Validation binary_cross_entropy = 0.344464
Epoch 257
Validation binary_cross_entropy = 0.359189
Epoch 258
Validation binary_cross_entropy = 0.372987
Epoch 259
Loss = 3.4321e-01, PNorm = 89.6574, GNorm = 36.3250, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.377915
Epoch 260
Validation binary_cross_entropy = 0.406948
Epoch 261
Validation binary_cross_entropy = 0.426974
Epoch 262
Validation binary_cross_entropy = 0.427256
Epoch 263
Validation binary_cross_entropy = 0.415105
Epoch 264
Loss = 4.2256e-01, PNorm = 89.7335, GNorm = 21.6266, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.408935
Epoch 265
Validation binary_cross_entropy = 0.414757
Epoch 266
Validation binary_cross_entropy = 0.422215
Epoch 267
Validation binary_cross_entropy = 0.438426
Epoch 268
Validation binary_cross_entropy = 0.453672
Epoch 269
Loss = 5.4021e-01, PNorm = 89.8336, GNorm = 43.0667, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.445168
Epoch 270
Validation binary_cross_entropy = 0.430265
Epoch 271
Validation binary_cross_entropy = 0.418943
Epoch 272
Validation binary_cross_entropy = 0.407105
Epoch 273
Validation binary_cross_entropy = 0.409111
Epoch 274
Loss = 3.2824e-01, PNorm = 89.8974, GNorm = 36.6016, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.393107
Epoch 275
Validation binary_cross_entropy = 0.386140
Epoch 276
Validation binary_cross_entropy = 0.395017
Epoch 277
Validation binary_cross_entropy = 0.442418
Epoch 278
Validation binary_cross_entropy = 0.549749
Epoch 279
Loss = 2.3256e-01, PNorm = 89.9998, GNorm = 35.7759, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.726063
Epoch 280
Validation binary_cross_entropy = 0.711899
Epoch 281
Validation binary_cross_entropy = 0.567080
Epoch 282
Validation binary_cross_entropy = 0.415874
Epoch 283
Validation binary_cross_entropy = 0.384108
Epoch 284
Loss = 3.2842e-01, PNorm = 90.0273, GNorm = 49.7219, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.408900
Epoch 285
Validation binary_cross_entropy = 0.451302
Epoch 286
Validation binary_cross_entropy = 0.449053
Epoch 287
Validation binary_cross_entropy = 0.463535
Epoch 288
Validation binary_cross_entropy = 0.489788
Epoch 289
Loss = 4.1428e-01, PNorm = 90.1126, GNorm = 46.6921, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.487051
Epoch 290
Validation binary_cross_entropy = 0.467215
Epoch 291
Validation binary_cross_entropy = 0.436375
Epoch 292
Validation binary_cross_entropy = 0.395952
Epoch 293
Validation binary_cross_entropy = 0.368594
Epoch 294
Loss = 5.8588e-01, PNorm = 90.1886, GNorm = 51.0015, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.381417
Epoch 295
Validation binary_cross_entropy = 0.427822
Epoch 296
Validation binary_cross_entropy = 0.516109
Epoch 297
Validation binary_cross_entropy = 0.645499
Epoch 298
Validation binary_cross_entropy = 0.695410
Epoch 299
Loss = 2.6944e-01, PNorm = 90.3145, GNorm = 17.0937, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.595496
Model 0 best validation binary_cross_entropy = 0.344464 on epoch 256
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.217674
Ensemble test binary_cross_entropy = 0.217674
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 3.915599
Epoch 1
Validation binary_cross_entropy = 3.114159
Epoch 2
Validation binary_cross_entropy = 85.876556
Epoch 3
Validation binary_cross_entropy = 5.155170
Epoch 4
Loss = 3.7352e+01, PNorm = 82.9715, GNorm = 1105.5055, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 28.084957
Epoch 5
Validation binary_cross_entropy = 63.321720
Epoch 6
Validation binary_cross_entropy = 6.403481
Epoch 7
Validation binary_cross_entropy = 57.732601
Epoch 8
Validation binary_cross_entropy = 10.578912
Epoch 9
Loss = 2.7075e+01, PNorm = 83.2555, GNorm = 559.9674, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 11.781337
Epoch 10
Validation binary_cross_entropy = 13.724245
Epoch 11
Validation binary_cross_entropy = 17.522215
Epoch 12
Validation binary_cross_entropy = 6.912455
Epoch 13
Validation binary_cross_entropy = 36.199371
Epoch 14
Loss = 2.2205e+01, PNorm = 83.6106, GNorm = 501.4968, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 28.390520
Epoch 15
Validation binary_cross_entropy = 8.788359
Epoch 16
Validation binary_cross_entropy = 15.194334
Epoch 17
Validation binary_cross_entropy = 22.860685
Epoch 18
Validation binary_cross_entropy = 9.241238
Epoch 19
Loss = 2.7263e+01, PNorm = 83.8683, GNorm = 451.3869, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 13.677375
Epoch 20
Validation binary_cross_entropy = 5.898666
Epoch 21
Validation binary_cross_entropy = 13.111035
Epoch 22
Validation binary_cross_entropy = 26.747953
Epoch 23
Validation binary_cross_entropy = 6.161726
Epoch 24
Loss = 2.4498e+01, PNorm = 84.0758, GNorm = 532.1539, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 5.753641
Epoch 25
Validation binary_cross_entropy = 19.148390
Epoch 26
Validation binary_cross_entropy = 29.192274
Epoch 27
Validation binary_cross_entropy = 6.143884
Epoch 28
Validation binary_cross_entropy = 5.388089
Epoch 29
Loss = 1.5862e+01, PNorm = 84.2462, GNorm = 344.1073, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 12.567887
Epoch 30
Validation binary_cross_entropy = 12.230966
Epoch 31
Validation binary_cross_entropy = 4.876812
Epoch 32
Validation binary_cross_entropy = 5.679127
Epoch 33
Validation binary_cross_entropy = 9.466929
Epoch 34
Loss = 7.7694e+00, PNorm = 84.3919, GNorm = 300.7201, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 21.730692
Epoch 35
Validation binary_cross_entropy = 6.869446
Epoch 36
Validation binary_cross_entropy = 5.239429
Epoch 37
Validation binary_cross_entropy = 4.992966
Epoch 38
Validation binary_cross_entropy = 4.786922
Epoch 39
Loss = 9.8316e+00, PNorm = 84.5035, GNorm = 309.8891, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 4.296386
Epoch 40
Validation binary_cross_entropy = 4.026088
Epoch 41
Validation binary_cross_entropy = 5.683094
Epoch 42
Validation binary_cross_entropy = 6.098658
Epoch 43
Validation binary_cross_entropy = 3.305607
Epoch 44
Loss = 6.9754e+00, PNorm = 84.5934, GNorm = 232.5111, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 4.053743
Epoch 45
Validation binary_cross_entropy = 10.121765
Epoch 46
Validation binary_cross_entropy = 5.715249
Epoch 47
Validation binary_cross_entropy = 3.032319
Epoch 48
Validation binary_cross_entropy = 2.769206
Epoch 49
Loss = 5.6282e+00, PNorm = 84.6714, GNorm = 223.7412, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.066236
Epoch 50
Validation binary_cross_entropy = 5.923250
Epoch 51
Validation binary_cross_entropy = 3.485904
Epoch 52
Validation binary_cross_entropy = 2.644432
Epoch 53
Validation binary_cross_entropy = 2.513518
Epoch 54
Loss = 4.5531e+00, PNorm = 84.7623, GNorm = 185.3919, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 7.139405
Epoch 55
Validation binary_cross_entropy = 10.412042
Epoch 56
Validation binary_cross_entropy = 8.132028
Epoch 57
Validation binary_cross_entropy = 3.173318
Epoch 58
Validation binary_cross_entropy = 5.078709
Epoch 59
Loss = 3.2253e+00, PNorm = 84.8472, GNorm = 137.2126, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 8.174306
Epoch 60
Validation binary_cross_entropy = 5.085065
Epoch 61
Validation binary_cross_entropy = 2.353754
Epoch 62
Validation binary_cross_entropy = 2.087204
Epoch 63
Validation binary_cross_entropy = 4.528464
Epoch 64
Loss = 5.9225e+00, PNorm = 84.9689, GNorm = 217.5222, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 3.168111
Epoch 65
Validation binary_cross_entropy = 3.118695
Epoch 66
Validation binary_cross_entropy = 2.568409
Epoch 67
Validation binary_cross_entropy = 1.977603
Epoch 68
Validation binary_cross_entropy = 1.263610
Epoch 69
Loss = 2.9417e+00, PNorm = 85.1647, GNorm = 218.8749, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 1.303556
Epoch 70
Validation binary_cross_entropy = 4.700067
Epoch 71
Validation binary_cross_entropy = 1.854407
Epoch 72
Validation binary_cross_entropy = 2.685337
Epoch 73
Validation binary_cross_entropy = 1.231561
Epoch 74
Loss = 1.8878e+00, PNorm = 85.2994, GNorm = 85.6296, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.117421
Epoch 75
Validation binary_cross_entropy = 1.691053
Epoch 76
Validation binary_cross_entropy = 1.874733
Epoch 77
Validation binary_cross_entropy = 3.838080
Epoch 78
Validation binary_cross_entropy = 2.641460
Epoch 79
Loss = 2.7976e+00, PNorm = 85.4183, GNorm = 91.8322, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.286063
Epoch 80
Validation binary_cross_entropy = 1.506221
Epoch 81
Validation binary_cross_entropy = 1.457835
Epoch 82
Validation binary_cross_entropy = 0.913766
Epoch 83
Validation binary_cross_entropy = 0.872019
Epoch 84
Loss = 1.1388e+00, PNorm = 85.5403, GNorm = 102.9041, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.898408
Epoch 85
Validation binary_cross_entropy = 2.054991
Epoch 86
Validation binary_cross_entropy = 0.787935
Epoch 87
Validation binary_cross_entropy = 0.879744
Epoch 88
Validation binary_cross_entropy = 0.808644
Epoch 89
Loss = 2.3697e+00, PNorm = 85.6101, GNorm = 68.4931, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.768310
Epoch 90
Validation binary_cross_entropy = 1.022760
Epoch 91
Validation binary_cross_entropy = 0.758953
Epoch 92
Validation binary_cross_entropy = 0.769993
Epoch 93
Validation binary_cross_entropy = 0.739364
Epoch 94
Loss = 1.8857e+00, PNorm = 85.6840, GNorm = 170.1946, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.063597
Epoch 95
Validation binary_cross_entropy = 1.827932
Epoch 96
Validation binary_cross_entropy = 0.918295
Epoch 97
Validation binary_cross_entropy = 0.714977
Epoch 98
Validation binary_cross_entropy = 0.712038
Epoch 99
Loss = 2.2517e+00, PNorm = 85.7851, GNorm = 139.5620, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.940897
Epoch 100
Validation binary_cross_entropy = 1.440085
Epoch 101
Validation binary_cross_entropy = 1.191032
Epoch 102
Validation binary_cross_entropy = 0.858447
Epoch 103
Validation binary_cross_entropy = 0.763642
Epoch 104
Loss = 2.0652e+00, PNorm = 85.8513, GNorm = 106.4108, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.804834
Epoch 105
Validation binary_cross_entropy = 0.891817
Epoch 106
Validation binary_cross_entropy = 1.016861
Epoch 107
Validation binary_cross_entropy = 0.937311
Epoch 108
Validation binary_cross_entropy = 0.871686
Epoch 109
Loss = 2.1522e+00, PNorm = 85.9167, GNorm = 88.1031, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.810440
Epoch 110
Validation binary_cross_entropy = 0.757772
Epoch 111
Validation binary_cross_entropy = 0.787913
Epoch 112
Validation binary_cross_entropy = 0.876437
Epoch 113
Validation binary_cross_entropy = 0.851143
Epoch 114
Loss = 2.5356e+00, PNorm = 86.0367, GNorm = 110.9940, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.702649
Epoch 115
Validation binary_cross_entropy = 0.589773
Epoch 116
Validation binary_cross_entropy = 0.556638
Epoch 117
Validation binary_cross_entropy = 0.535775
Epoch 118
Validation binary_cross_entropy = 0.523958
Epoch 119
Loss = 1.0067e+00, PNorm = 86.1566, GNorm = 33.5406, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.559229
Epoch 120
Validation binary_cross_entropy = 0.638078
Epoch 121
Validation binary_cross_entropy = 0.526656
Epoch 122
Validation binary_cross_entropy = 0.527931
Epoch 123
Validation binary_cross_entropy = 0.523371
Epoch 124
Loss = 1.1917e+00, PNorm = 86.2355, GNorm = 67.3681, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.536049
Epoch 125
Validation binary_cross_entropy = 0.525033
Epoch 126
Validation binary_cross_entropy = 0.633672
Epoch 127
Validation binary_cross_entropy = 0.668783
Epoch 128
Validation binary_cross_entropy = 0.518012
Epoch 129
Loss = 9.3869e-01, PNorm = 86.3414, GNorm = 71.4293, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.494986
Epoch 130
Validation binary_cross_entropy = 0.479881
Epoch 131
Validation binary_cross_entropy = 0.477659
Epoch 132
Validation binary_cross_entropy = 0.508888
Epoch 133
Validation binary_cross_entropy = 0.539095
Epoch 134
Loss = 1.1151e+00, PNorm = 86.4749, GNorm = 53.3116, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.595656
Epoch 135
Validation binary_cross_entropy = 0.581409
Epoch 136
Validation binary_cross_entropy = 0.519169
Epoch 137
Validation binary_cross_entropy = 0.474124
Epoch 138
Validation binary_cross_entropy = 0.470696
Epoch 139
Loss = 5.3933e-01, PNorm = 86.5682, GNorm = 32.5298, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.453599
Epoch 140
Validation binary_cross_entropy = 0.446125
Epoch 141
Validation binary_cross_entropy = 0.464393
Epoch 142
Validation binary_cross_entropy = 0.529740
Epoch 143
Validation binary_cross_entropy = 0.588782
Epoch 144
Loss = 7.8916e-01, PNorm = 86.6615, GNorm = 31.5133, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.561768
Epoch 145
Validation binary_cross_entropy = 0.549227
Epoch 146
Validation binary_cross_entropy = 0.511542
Epoch 147
Validation binary_cross_entropy = 0.461459
Epoch 148
Validation binary_cross_entropy = 0.463774
Epoch 149
Loss = 7.9312e-01, PNorm = 86.7339, GNorm = 55.8495, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.476208
Epoch 150
Validation binary_cross_entropy = 0.519916
Epoch 151
Validation binary_cross_entropy = 0.518164
Epoch 152
Validation binary_cross_entropy = 0.525818
Epoch 153
Validation binary_cross_entropy = 0.541194
Epoch 154
Loss = 1.0208e+00, PNorm = 86.8172, GNorm = 48.0708, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.469357
Epoch 155
Validation binary_cross_entropy = 0.407252
Epoch 156
Validation binary_cross_entropy = 0.394907
Epoch 157
Validation binary_cross_entropy = 0.406494
Epoch 158
Validation binary_cross_entropy = 0.541200
Epoch 159
Loss = 5.8302e-01, PNorm = 86.9374, GNorm = 76.4872, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.766327
Epoch 160
Validation binary_cross_entropy = 0.849936
Epoch 161
Validation binary_cross_entropy = 0.847274
Epoch 162
Validation binary_cross_entropy = 0.477457
Epoch 163
Validation binary_cross_entropy = 0.403237
Epoch 164
Loss = 2.9865e-01, PNorm = 87.0490, GNorm = 50.0212, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.403606
Epoch 165
Validation binary_cross_entropy = 0.404939
Epoch 166
Validation binary_cross_entropy = 0.461016
Epoch 167
Validation binary_cross_entropy = 0.557833
Epoch 168
Validation binary_cross_entropy = 0.531093
Epoch 169
Loss = 2.5935e-01, PNorm = 87.1515, GNorm = 19.8654, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.485986
Epoch 170
Validation binary_cross_entropy = 0.466368
Epoch 171
Validation binary_cross_entropy = 0.428535
Epoch 172
Validation binary_cross_entropy = 0.390788
Epoch 173
Validation binary_cross_entropy = 0.384430
Epoch 174
Loss = 5.3577e-01, PNorm = 87.2507, GNorm = 33.1411, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.387370
Epoch 175
Validation binary_cross_entropy = 0.430235
Epoch 176
Validation binary_cross_entropy = 0.505625
Epoch 177
Validation binary_cross_entropy = 0.560761
Epoch 178
Validation binary_cross_entropy = 0.514900
Epoch 179
Loss = 8.2710e-01, PNorm = 87.3670, GNorm = 57.1399, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.382897
Epoch 180
Validation binary_cross_entropy = 0.362625
Epoch 181
Validation binary_cross_entropy = 0.368953
Epoch 182
Validation binary_cross_entropy = 0.410569
Epoch 183
Validation binary_cross_entropy = 0.499220
Epoch 184
Loss = 7.3068e-01, PNorm = 87.4767, GNorm = 59.5587, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.582157
Epoch 185
Validation binary_cross_entropy = 0.577974
Epoch 186
Validation binary_cross_entropy = 0.493981
Epoch 187
Validation binary_cross_entropy = 0.436506
Epoch 188
Validation binary_cross_entropy = 0.405008
Epoch 189
Loss = 4.9886e-01, PNorm = 87.5928, GNorm = 32.4017, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.381677
Epoch 190
Validation binary_cross_entropy = 0.377887
Epoch 191
Validation binary_cross_entropy = 0.385966
Epoch 192
Validation binary_cross_entropy = 0.405358
Epoch 193
Validation binary_cross_entropy = 0.413606
Epoch 194
Loss = 5.0446e-01, PNorm = 87.7190, GNorm = 34.7865, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.409839
Epoch 195
Validation binary_cross_entropy = 0.381716
Epoch 196
Validation binary_cross_entropy = 0.360487
Epoch 197
Validation binary_cross_entropy = 0.352047
Epoch 198
Validation binary_cross_entropy = 0.347162
Epoch 199
Loss = 2.0614e-01, PNorm = 87.8041, GNorm = 26.3954, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.352146
Epoch 200
Validation binary_cross_entropy = 0.350174
Epoch 201
Validation binary_cross_entropy = 0.347149
Epoch 202
Validation binary_cross_entropy = 0.342800
Epoch 203
Validation binary_cross_entropy = 0.348607
Epoch 204
Loss = 4.6423e-01, PNorm = 87.8690, GNorm = 28.6984, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.357762
Epoch 205
Validation binary_cross_entropy = 0.383659
Epoch 206
Validation binary_cross_entropy = 0.403400
Epoch 207
Validation binary_cross_entropy = 0.379897
Epoch 208
Validation binary_cross_entropy = 0.347722
Epoch 209
Loss = 4.3278e-01, PNorm = 87.9368, GNorm = 59.2383, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.327910
Epoch 210
Validation binary_cross_entropy = 0.328380
Epoch 211
Validation binary_cross_entropy = 0.339602
Epoch 212
Validation binary_cross_entropy = 0.370332
Epoch 213
Validation binary_cross_entropy = 0.414114
Epoch 214
Loss = 4.3016e-01, PNorm = 88.0454, GNorm = 33.5419, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.445631
Epoch 215
Validation binary_cross_entropy = 0.473438
Epoch 216
Validation binary_cross_entropy = 0.446788
Epoch 217
Validation binary_cross_entropy = 0.386508
Epoch 218
Validation binary_cross_entropy = 0.375240
Epoch 219
Loss = 2.7907e-01, PNorm = 88.1868, GNorm = 41.0377, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.416657
Epoch 220
Validation binary_cross_entropy = 0.491235
Epoch 221
Validation binary_cross_entropy = 0.530362
Epoch 222
Validation binary_cross_entropy = 0.476133
Epoch 223
Validation binary_cross_entropy = 0.402263
Epoch 224
Loss = 2.2867e-01, PNorm = 88.3253, GNorm = 19.2060, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.376516
Epoch 225
Validation binary_cross_entropy = 0.369888
Epoch 226
Validation binary_cross_entropy = 0.385507
Epoch 227
Validation binary_cross_entropy = 0.395210
Epoch 228
Validation binary_cross_entropy = 0.377960
Epoch 229
Loss = 2.0080e-01, PNorm = 88.4432, GNorm = 28.8034, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.381062
Epoch 230
Validation binary_cross_entropy = 0.409383
Epoch 231
Validation binary_cross_entropy = 0.428216
Epoch 232
Validation binary_cross_entropy = 0.405839
Epoch 233
Validation binary_cross_entropy = 0.374894
Epoch 234
Loss = 3.3219e-01, PNorm = 88.5798, GNorm = 33.9624, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.356361
Epoch 235
Validation binary_cross_entropy = 0.340607
Epoch 236
Validation binary_cross_entropy = 0.334207
Epoch 237
Validation binary_cross_entropy = 0.337329
Epoch 238
Validation binary_cross_entropy = 0.376112
Epoch 239
Loss = 3.1817e-01, PNorm = 88.7260, GNorm = 50.5818, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.396333
Epoch 240
Validation binary_cross_entropy = 0.386208
Epoch 241
Validation binary_cross_entropy = 0.359256
Epoch 242
Validation binary_cross_entropy = 0.346213
Epoch 243
Validation binary_cross_entropy = 0.356651
Epoch 244
Loss = 3.0376e-01, PNorm = 88.8600, GNorm = 25.1485, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.374557
Epoch 245
Validation binary_cross_entropy = 0.424391
Epoch 246
Validation binary_cross_entropy = 0.477983
Epoch 247
Validation binary_cross_entropy = 0.504486
Epoch 248
Validation binary_cross_entropy = 0.399344
Epoch 249
Loss = 2.0081e-01, PNorm = 88.9803, GNorm = 52.3033, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.351284
Epoch 250
Validation binary_cross_entropy = 0.344059
Epoch 251
Validation binary_cross_entropy = 0.370612
Epoch 252
Validation binary_cross_entropy = 0.408439
Epoch 253
Validation binary_cross_entropy = 0.409821
Epoch 254
Loss = 3.2024e-01, PNorm = 89.0818, GNorm = 31.9334, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.402325
Epoch 255
Validation binary_cross_entropy = 0.385857
Epoch 256
Validation binary_cross_entropy = 0.383106
Epoch 257
Validation binary_cross_entropy = 0.383567
Epoch 258
Validation binary_cross_entropy = 0.401326
Epoch 259
Loss = 2.6499e-01, PNorm = 89.2131, GNorm = 11.7790, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.408368
Epoch 260
Validation binary_cross_entropy = 0.386369
Epoch 261
Validation binary_cross_entropy = 0.376557
Epoch 262
Validation binary_cross_entropy = 0.386503
Epoch 263
Validation binary_cross_entropy = 0.395283
Epoch 264
Loss = 3.7996e-01, PNorm = 89.3920, GNorm = 61.5935, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.396916
Epoch 265
Validation binary_cross_entropy = 0.407835
Epoch 266
Validation binary_cross_entropy = 0.494012
Epoch 267
Validation binary_cross_entropy = 0.478734
Epoch 268
Validation binary_cross_entropy = 0.358485
Epoch 269
Loss = 1.6654e-01, PNorm = 89.5262, GNorm = 10.1403, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.331081
Epoch 270
Validation binary_cross_entropy = 0.328762
Epoch 271
Validation binary_cross_entropy = 0.333962
Epoch 272
Validation binary_cross_entropy = 0.354751
Epoch 273
Validation binary_cross_entropy = 0.377881
Epoch 274
Loss = 2.2395e-01, PNorm = 89.7183, GNorm = 27.1977, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.381321
Epoch 275
Validation binary_cross_entropy = 0.372999
Epoch 276
Validation binary_cross_entropy = 0.377862
Epoch 277
Validation binary_cross_entropy = 0.380436
Epoch 278
Validation binary_cross_entropy = 0.378668
Epoch 279
Loss = 1.3588e-01, PNorm = 89.8464, GNorm = 17.4949, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.376665
Epoch 280
Validation binary_cross_entropy = 0.380651
Epoch 281
Validation binary_cross_entropy = 0.410498
Epoch 282
Validation binary_cross_entropy = 0.407193
Epoch 283
Validation binary_cross_entropy = 0.365265
Epoch 284
Loss = 2.5158e-01, PNorm = 89.9320, GNorm = 29.1669, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.352161
Epoch 285
Validation binary_cross_entropy = 0.358623
Epoch 286
Validation binary_cross_entropy = 0.406509
Epoch 287
Validation binary_cross_entropy = 0.464317
Epoch 288
Validation binary_cross_entropy = 0.438857
Epoch 289
Loss = 4.6623e-01, PNorm = 90.0319, GNorm = 36.8119, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.371248
Epoch 290
Validation binary_cross_entropy = 0.330532
Epoch 291
Validation binary_cross_entropy = 0.323408
Epoch 292
Validation binary_cross_entropy = 0.321386
Epoch 293
Validation binary_cross_entropy = 0.326037
Epoch 294
Loss = 1.1135e-01, PNorm = 90.1189, GNorm = 10.7105, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.334255
Epoch 295
Validation binary_cross_entropy = 0.352920
Epoch 296
Validation binary_cross_entropy = 0.403085
Epoch 297
Validation binary_cross_entropy = 0.401332
Epoch 298
Validation binary_cross_entropy = 0.348539
Epoch 299
Loss = 2.4068e-01, PNorm = 90.2251, GNorm = 25.1251, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.332731
Model 0 best validation binary_cross_entropy = 0.321386 on epoch 292
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.170909
Ensemble test binary_cross_entropy = 0.170909
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 65.966423
Epoch 1
Validation binary_cross_entropy = 9.021070
Epoch 2
Validation binary_cross_entropy = 82.847061
Epoch 3
Validation binary_cross_entropy = 8.564458
Epoch 4
Loss = 3.4464e+01, PNorm = 82.9657, GNorm = 724.2070, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 7.098057
Epoch 5
Validation binary_cross_entropy = 44.936668
Epoch 6
Validation binary_cross_entropy = 83.163727
Epoch 7
Validation binary_cross_entropy = 6.785495
Epoch 8
Validation binary_cross_entropy = 6.999262
Epoch 9
Loss = 3.1383e+01, PNorm = 83.2411, GNorm = 542.6165, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 69.938889
Epoch 10
Validation binary_cross_entropy = 37.430233
Epoch 11
Validation binary_cross_entropy = 6.739348
Epoch 12
Validation binary_cross_entropy = 7.721876
Epoch 13
Validation binary_cross_entropy = 43.420433
Epoch 14
Loss = 2.0994e+01, PNorm = 83.5404, GNorm = 649.5861, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 29.467897
Epoch 15
Validation binary_cross_entropy = 6.131789
Epoch 16
Validation binary_cross_entropy = 8.359489
Epoch 17
Validation binary_cross_entropy = 34.963924
Epoch 18
Validation binary_cross_entropy = 22.346083
Epoch 19
Loss = 1.0173e+01, PNorm = 83.7509, GNorm = 310.0234, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 9.995477
Epoch 20
Validation binary_cross_entropy = 5.958771
Epoch 21
Validation binary_cross_entropy = 14.341572
Epoch 22
Validation binary_cross_entropy = 15.859574
Epoch 23
Validation binary_cross_entropy = 5.325609
Epoch 24
Loss = 1.1935e+01, PNorm = 83.8878, GNorm = 383.1043, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 5.790407
Epoch 25
Validation binary_cross_entropy = 8.970201
Epoch 26
Validation binary_cross_entropy = 5.018726
Epoch 27
Validation binary_cross_entropy = 5.017776
Epoch 28
Validation binary_cross_entropy = 5.537702
Epoch 29
Loss = 9.2141e+00, PNorm = 83.9876, GNorm = 269.9573, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 13.803739
Epoch 30
Validation binary_cross_entropy = 5.387185
Epoch 31
Validation binary_cross_entropy = 4.595804
Epoch 32
Validation binary_cross_entropy = 4.519351
Epoch 33
Validation binary_cross_entropy = 9.270292
Epoch 34
Loss = 1.9639e+01, PNorm = 84.0786, GNorm = 294.0338, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 4.804508
Epoch 35
Validation binary_cross_entropy = 3.180085
Epoch 36
Validation binary_cross_entropy = 5.204051
Epoch 37
Validation binary_cross_entropy = 6.085668
Epoch 38
Validation binary_cross_entropy = 27.314760
Epoch 39
Loss = 1.1958e+01, PNorm = 84.1896, GNorm = 336.8555, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 6.187265
Epoch 40
Validation binary_cross_entropy = 3.035636
Epoch 41
Validation binary_cross_entropy = 3.169767
Epoch 42
Validation binary_cross_entropy = 2.995780
Epoch 43
Validation binary_cross_entropy = 3.042147
Epoch 44
Loss = 5.4130e+00, PNorm = 84.2974, GNorm = 301.8243, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 3.125464
Epoch 45
Validation binary_cross_entropy = 3.738194
Epoch 46
Validation binary_cross_entropy = 2.803191
Epoch 47
Validation binary_cross_entropy = 2.648415
Epoch 48
Validation binary_cross_entropy = 2.402842
Epoch 49
Loss = 3.5468e+00, PNorm = 84.4012, GNorm = 95.5390, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 2.705437
Epoch 50
Validation binary_cross_entropy = 2.489820
Epoch 51
Validation binary_cross_entropy = 2.548042
Epoch 52
Validation binary_cross_entropy = 2.324309
Epoch 53
Validation binary_cross_entropy = 2.439988
Epoch 54
Loss = 6.1882e+00, PNorm = 84.4705, GNorm = 194.7868, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 2.580640
Epoch 55
Validation binary_cross_entropy = 2.172252
Epoch 56
Validation binary_cross_entropy = 1.963284
Epoch 57
Validation binary_cross_entropy = 1.628349
Epoch 58
Validation binary_cross_entropy = 1.771538
Epoch 59
Loss = 5.7666e+00, PNorm = 84.5279, GNorm = 126.6804, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 1.955957
Epoch 60
Validation binary_cross_entropy = 1.536026
Epoch 61
Validation binary_cross_entropy = 1.724783
Epoch 62
Validation binary_cross_entropy = 1.852315
Epoch 63
Validation binary_cross_entropy = 1.257317
Epoch 64
Loss = 7.9166e+00, PNorm = 84.5792, GNorm = 328.8848, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 1.815073
Epoch 65
Validation binary_cross_entropy = 1.378679
Epoch 66
Validation binary_cross_entropy = 1.385924
Epoch 67
Validation binary_cross_entropy = 1.657279
Epoch 68
Validation binary_cross_entropy = 2.437343
Epoch 69
Loss = 3.1362e+00, PNorm = 84.6305, GNorm = 131.6271, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.122861
Epoch 70
Validation binary_cross_entropy = 1.087974
Epoch 71
Validation binary_cross_entropy = 1.214481
Epoch 72
Validation binary_cross_entropy = 2.210729
Epoch 73
Validation binary_cross_entropy = 3.972441
Epoch 74
Loss = 2.9795e+00, PNorm = 84.6828, GNorm = 89.9005, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.024644
Epoch 75
Validation binary_cross_entropy = 1.218263
Epoch 76
Validation binary_cross_entropy = 1.267875
Epoch 77
Validation binary_cross_entropy = 1.842335
Epoch 78
Validation binary_cross_entropy = 6.183700
Epoch 79
Loss = 5.2679e+00, PNorm = 84.7424, GNorm = 186.2801, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 3.958372
Epoch 80
Validation binary_cross_entropy = 0.979853
Epoch 81
Validation binary_cross_entropy = 1.222154
Epoch 82
Validation binary_cross_entropy = 1.013009
Epoch 83
Validation binary_cross_entropy = 1.022290
Epoch 84
Loss = 2.9252e+00, PNorm = 84.7870, GNorm = 111.4479, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.025362
Epoch 85
Validation binary_cross_entropy = 0.883699
Epoch 86
Validation binary_cross_entropy = 0.862449
Epoch 87
Validation binary_cross_entropy = 1.719257
Epoch 88
Validation binary_cross_entropy = 3.049907
Epoch 89
Loss = 4.1243e+00, PNorm = 84.8484, GNorm = 261.8880, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 2.516455
Epoch 90
Validation binary_cross_entropy = 0.774859
Epoch 91
Validation binary_cross_entropy = 0.814077
Epoch 92
Validation binary_cross_entropy = 0.798278
Epoch 93
Validation binary_cross_entropy = 1.103889
Epoch 94
Loss = 1.9682e+00, PNorm = 84.9217, GNorm = 140.2437, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 2.691142
Epoch 95
Validation binary_cross_entropy = 2.647928
Epoch 96
Validation binary_cross_entropy = 1.058929
Epoch 97
Validation binary_cross_entropy = 0.917081
Epoch 98
Validation binary_cross_entropy = 0.899006
Epoch 99
Loss = 1.4935e+00, PNorm = 84.9597, GNorm = 77.2121, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.896014
Epoch 100
Validation binary_cross_entropy = 0.855201
Epoch 101
Validation binary_cross_entropy = 0.899972
Epoch 102
Validation binary_cross_entropy = 0.827309
Epoch 103
Validation binary_cross_entropy = 0.842696
Epoch 104
Loss = 1.9818e+00, PNorm = 85.0214, GNorm = 98.1293, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.855273
Epoch 105
Validation binary_cross_entropy = 0.748270
Epoch 106
Validation binary_cross_entropy = 0.861554
Epoch 107
Validation binary_cross_entropy = 0.962352
Epoch 108
Validation binary_cross_entropy = 0.712114
Epoch 109
Loss = 7.6838e-01, PNorm = 85.0832, GNorm = 88.5271, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.685061
Epoch 110
Validation binary_cross_entropy = 0.654168
Epoch 111
Validation binary_cross_entropy = 0.650509
Epoch 112
Validation binary_cross_entropy = 0.657596
Epoch 113
Validation binary_cross_entropy = 0.714448
Epoch 114
Loss = 1.2302e+00, PNorm = 85.1697, GNorm = 89.0667, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.864533
Epoch 115
Validation binary_cross_entropy = 1.399712
Epoch 116
Validation binary_cross_entropy = 0.630983
Epoch 117
Validation binary_cross_entropy = 0.529959
Epoch 118
Validation binary_cross_entropy = 0.554055
Epoch 119
Loss = 1.4834e+00, PNorm = 85.2620, GNorm = 87.9173, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.527068
Epoch 120
Validation binary_cross_entropy = 0.516935
Epoch 121
Validation binary_cross_entropy = 0.637050
Epoch 122
Validation binary_cross_entropy = 0.692435
Epoch 123
Validation binary_cross_entropy = 0.531202
Epoch 124
Loss = 6.9285e-01, PNorm = 85.3725, GNorm = 47.0621, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.451062
Epoch 125
Validation binary_cross_entropy = 0.431282
Epoch 126
Validation binary_cross_entropy = 0.420183
Epoch 127
Validation binary_cross_entropy = 0.407528
Epoch 128
Validation binary_cross_entropy = 0.380574
Epoch 129
Loss = 8.8923e-01, PNorm = 85.4629, GNorm = 64.4248, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.431953
Epoch 130
Validation binary_cross_entropy = 0.436342
Epoch 131
Validation binary_cross_entropy = 0.406336
Epoch 132
Validation binary_cross_entropy = 0.652253
Epoch 133
Validation binary_cross_entropy = 1.050528
Epoch 134
Loss = 2.3966e+00, PNorm = 85.5649, GNorm = 86.0111, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.712498
Epoch 135
Validation binary_cross_entropy = 0.483523
Epoch 136
Validation binary_cross_entropy = 0.439386
Epoch 137
Validation binary_cross_entropy = 0.436307
Epoch 138
Validation binary_cross_entropy = 0.420293
Epoch 139
Loss = 9.9321e-01, PNorm = 85.6678, GNorm = 35.6625, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.432370
Epoch 140
Validation binary_cross_entropy = 0.429797
Epoch 141
Validation binary_cross_entropy = 0.405719
Epoch 142
Validation binary_cross_entropy = 0.384568
Epoch 143
Validation binary_cross_entropy = 0.375569
Epoch 144
Loss = 1.0403e+00, PNorm = 85.7557, GNorm = 55.0206, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.373831
Epoch 145
Validation binary_cross_entropy = 0.391717
Epoch 146
Validation binary_cross_entropy = 0.450255
Epoch 147
Validation binary_cross_entropy = 0.460758
Epoch 148
Validation binary_cross_entropy = 0.416168
Epoch 149
Loss = 5.7346e-01, PNorm = 85.9051, GNorm = 38.9974, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.425001
Epoch 150
Validation binary_cross_entropy = 0.477804
Epoch 151
Validation binary_cross_entropy = 0.520416
Epoch 152
Validation binary_cross_entropy = 0.446795
Epoch 153
Validation binary_cross_entropy = 0.422687
Epoch 154
Loss = 5.9636e-01, PNorm = 86.0126, GNorm = 30.8164, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.362941
Epoch 155
Validation binary_cross_entropy = 0.347016
Epoch 156
Validation binary_cross_entropy = 0.345915
Epoch 157
Validation binary_cross_entropy = 0.365417
Epoch 158
Validation binary_cross_entropy = 0.454689
Epoch 159
Loss = 6.6072e-01, PNorm = 86.1219, GNorm = 68.1491, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.558154
Epoch 160
Validation binary_cross_entropy = 0.587822
Epoch 161
Validation binary_cross_entropy = 0.496305
Epoch 162
Validation binary_cross_entropy = 0.423182
Epoch 163
Validation binary_cross_entropy = 0.392566
Epoch 164
Loss = 8.1501e-01, PNorm = 86.2193, GNorm = 58.7699, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.359765
Epoch 165
Validation binary_cross_entropy = 0.359065
Epoch 166
Validation binary_cross_entropy = 0.369381
Epoch 167
Validation binary_cross_entropy = 0.434976
Epoch 168
Validation binary_cross_entropy = 0.522584
Epoch 169
Loss = 7.6124e-01, PNorm = 86.3068, GNorm = 50.0243, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.512653
Epoch 170
Validation binary_cross_entropy = 0.427418
Epoch 171
Validation binary_cross_entropy = 0.356766
Epoch 172
Validation binary_cross_entropy = 0.346836
Epoch 173
Validation binary_cross_entropy = 0.350604
Epoch 174
Loss = 6.8467e-01, PNorm = 86.3858, GNorm = 67.0130, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.366094
Epoch 175
Validation binary_cross_entropy = 0.400987
Epoch 176
Validation binary_cross_entropy = 0.424385
Epoch 177
Validation binary_cross_entropy = 0.431458
Epoch 178
Validation binary_cross_entropy = 0.400707
Epoch 179
Loss = 7.2888e-01, PNorm = 86.4728, GNorm = 65.2994, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.366747
Epoch 180
Validation binary_cross_entropy = 0.355688
Epoch 181
Validation binary_cross_entropy = 0.358308
Epoch 182
Validation binary_cross_entropy = 0.374671
Epoch 183
Validation binary_cross_entropy = 0.434526
Epoch 184
Loss = 7.6141e-01, PNorm = 86.5839, GNorm = 57.5773, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.494230
Epoch 185
Validation binary_cross_entropy = 0.458811
Epoch 186
Validation binary_cross_entropy = 0.385886
Epoch 187
Validation binary_cross_entropy = 0.361638
Epoch 188
Validation binary_cross_entropy = 0.376903
Epoch 189
Loss = 8.5248e-01, PNorm = 86.6750, GNorm = 28.1340, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.368124
Epoch 190
Validation binary_cross_entropy = 0.363986
Epoch 191
Validation binary_cross_entropy = 0.405147
Epoch 192
Validation binary_cross_entropy = 0.481150
Epoch 193
Validation binary_cross_entropy = 0.553550
Epoch 194
Loss = 7.2802e-01, PNorm = 86.8257, GNorm = 51.2471, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.525253
Epoch 195
Validation binary_cross_entropy = 0.440794
Epoch 196
Validation binary_cross_entropy = 0.376588
Epoch 197
Validation binary_cross_entropy = 0.359876
Epoch 198
Validation binary_cross_entropy = 0.358879
Epoch 199
Loss = 3.6928e-01, PNorm = 86.9375, GNorm = 38.1716, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.365453
Epoch 200
Validation binary_cross_entropy = 0.375190
Epoch 201
Validation binary_cross_entropy = 0.399912
Epoch 202
Validation binary_cross_entropy = 0.426357
Epoch 203
Validation binary_cross_entropy = 0.442301
Epoch 204
Loss = 2.4288e-01, PNorm = 87.0638, GNorm = 22.8438, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.434590
Epoch 205
Validation binary_cross_entropy = 0.411081
Epoch 206
Validation binary_cross_entropy = 0.380200
Epoch 207
Validation binary_cross_entropy = 0.369576
Epoch 208
Validation binary_cross_entropy = 0.368389
Epoch 209
Loss = 4.4212e-01, PNorm = 87.1497, GNorm = 26.9062, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.371079
Epoch 210
Validation binary_cross_entropy = 0.373908
Epoch 211
Validation binary_cross_entropy = 0.365270
Epoch 212
Validation binary_cross_entropy = 0.368767
Epoch 213
Validation binary_cross_entropy = 0.384687
Epoch 214
Loss = 4.5119e-01, PNorm = 87.2548, GNorm = 25.4194, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.407797
Epoch 215
Validation binary_cross_entropy = 0.415248
Epoch 216
Validation binary_cross_entropy = 0.409229
Epoch 217
Validation binary_cross_entropy = 0.406084
Epoch 218
Validation binary_cross_entropy = 0.403687
Epoch 219
Loss = 4.4559e-01, PNorm = 87.3795, GNorm = 18.4535, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.387858
Epoch 220
Validation binary_cross_entropy = 0.377257
Epoch 221
Validation binary_cross_entropy = 0.373589
Epoch 222
Validation binary_cross_entropy = 0.372854
Epoch 223
Validation binary_cross_entropy = 0.363180
Epoch 224
Loss = 6.0912e-01, PNorm = 87.4987, GNorm = 34.8333, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.357677
Epoch 225
Validation binary_cross_entropy = 0.378266
Epoch 226
Validation binary_cross_entropy = 0.399411
Epoch 227
Validation binary_cross_entropy = 0.392598
Epoch 228
Validation binary_cross_entropy = 0.377439
Epoch 229
Loss = 5.5390e-01, PNorm = 87.6299, GNorm = 37.7019, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.352254
Epoch 230
Validation binary_cross_entropy = 0.330887
Epoch 231
Validation binary_cross_entropy = 0.334196
Epoch 232
Validation binary_cross_entropy = 0.369869
Epoch 233
Validation binary_cross_entropy = 0.422618
Epoch 234
Loss = 5.7646e-01, PNorm = 87.8205, GNorm = 38.3768, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.442773
Epoch 235
Validation binary_cross_entropy = 0.372201
Epoch 236
Validation binary_cross_entropy = 0.332002
Epoch 237
Validation binary_cross_entropy = 0.320341
Epoch 238
Validation binary_cross_entropy = 0.334382
Epoch 239
Loss = 4.6246e-01, PNorm = 87.9997, GNorm = 26.5823, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.368664
Epoch 240
Validation binary_cross_entropy = 0.401096
Epoch 241
Validation binary_cross_entropy = 0.385821
Epoch 242
Validation binary_cross_entropy = 0.361776
Epoch 243
Validation binary_cross_entropy = 0.331439
Epoch 244
Loss = 5.0868e-01, PNorm = 88.1577, GNorm = 31.0316, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.319504
Epoch 245
Validation binary_cross_entropy = 0.321098
Epoch 246
Validation binary_cross_entropy = 0.335478
Epoch 247
Validation binary_cross_entropy = 0.355269
Epoch 248
Validation binary_cross_entropy = 0.351286
Epoch 249
Loss = 4.1804e-01, PNorm = 88.3082, GNorm = 26.4174, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.343749
Epoch 250
Validation binary_cross_entropy = 0.338901
Epoch 251
Validation binary_cross_entropy = 0.339936
Epoch 252
Validation binary_cross_entropy = 0.342389
Epoch 253
Validation binary_cross_entropy = 0.349653
Epoch 254
Loss = 3.6932e-01, PNorm = 88.4382, GNorm = 25.9266, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.354160
Epoch 255
Validation binary_cross_entropy = 0.372725
Epoch 256
Validation binary_cross_entropy = 0.369836
Epoch 257
Validation binary_cross_entropy = 0.365723
Epoch 258
Validation binary_cross_entropy = 0.360164
Epoch 259
Loss = 3.5990e-01, PNorm = 88.5412, GNorm = 29.3567, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.343060
Epoch 260
Validation binary_cross_entropy = 0.329730
Epoch 261
Validation binary_cross_entropy = 0.322607
Epoch 262
Validation binary_cross_entropy = 0.326019
Epoch 263
Validation binary_cross_entropy = 0.338293
Epoch 264
Loss = 3.9122e-01, PNorm = 88.6607, GNorm = 24.7502, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.358764
Epoch 265
Validation binary_cross_entropy = 0.362278
Epoch 266
Validation binary_cross_entropy = 0.367427
Epoch 267
Validation binary_cross_entropy = 0.381799
Epoch 268
Validation binary_cross_entropy = 0.376925
Epoch 269
Loss = 4.8111e-01, PNorm = 88.7428, GNorm = 21.7044, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.332862
Epoch 270
Validation binary_cross_entropy = 0.301555
Epoch 271
Validation binary_cross_entropy = 0.291534
Epoch 272
Validation binary_cross_entropy = 0.303601
Epoch 273
Validation binary_cross_entropy = 0.336547
Epoch 274
Loss = 3.4491e-01, PNorm = 88.8605, GNorm = 31.1753, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.362318
Epoch 275
Validation binary_cross_entropy = 0.386486
Epoch 276
Validation binary_cross_entropy = 0.393003
Epoch 277
Validation binary_cross_entropy = 0.385954
Epoch 278
Validation binary_cross_entropy = 0.382261
Epoch 279
Loss = 4.4835e-01, PNorm = 89.0092, GNorm = 17.9225, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.354891
Epoch 280
Validation binary_cross_entropy = 0.317899
Epoch 281
Validation binary_cross_entropy = 0.305237
Epoch 282
Validation binary_cross_entropy = 0.304166
Epoch 283
Validation binary_cross_entropy = 0.305810
Epoch 284
Loss = 3.3979e-01, PNorm = 89.0802, GNorm = 30.1722, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.313925
Epoch 285
Validation binary_cross_entropy = 0.329064
Epoch 286
Validation binary_cross_entropy = 0.354661
Epoch 287
Validation binary_cross_entropy = 0.394215
Epoch 288
Validation binary_cross_entropy = 0.416255
Epoch 289
Loss = 2.5087e-01, PNorm = 89.1962, GNorm = 42.5390, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.381000
Epoch 290
Validation binary_cross_entropy = 0.366020
Epoch 291
Validation binary_cross_entropy = 0.343740
Epoch 292
Validation binary_cross_entropy = 0.329670
Epoch 293
Validation binary_cross_entropy = 0.324265
Epoch 294
Loss = 3.1493e-01, PNorm = 89.2802, GNorm = 34.6269, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.334062
Epoch 295
Validation binary_cross_entropy = 0.362096
Epoch 296
Validation binary_cross_entropy = 0.395293
Epoch 297
Validation binary_cross_entropy = 0.393324
Epoch 298
Validation binary_cross_entropy = 0.349432
Epoch 299
Loss = 6.0731e-01, PNorm = 89.3683, GNorm = 36.3633, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.316130
Model 0 best validation binary_cross_entropy = 0.291534 on epoch 271
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.162515
Ensemble test binary_cross_entropy = 0.162515
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 44.800652
Epoch 1
Validation binary_cross_entropy = 5.374447
Epoch 2
Validation binary_cross_entropy = 87.105621
Epoch 3
Validation binary_cross_entropy = 5.692735
Epoch 4
Loss = 4.3030e+01, PNorm = 82.9645, GNorm = 1167.4250, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 5.479221
Epoch 5
Validation binary_cross_entropy = 56.054947
Epoch 6
Validation binary_cross_entropy = 6.323285
Epoch 7
Validation binary_cross_entropy = 16.745060
Epoch 8
Validation binary_cross_entropy = 17.104912
Epoch 9
Loss = 3.9483e+01, PNorm = 83.2445, GNorm = 638.1503, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 9.659070
Epoch 10
Validation binary_cross_entropy = 34.435352
Epoch 11
Validation binary_cross_entropy = 22.077980
Epoch 12
Validation binary_cross_entropy = 34.879799
Epoch 13
Validation binary_cross_entropy = 19.350531
Epoch 14
Loss = 2.0362e+01, PNorm = 83.5899, GNorm = 575.8976, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 10.227946
Epoch 15
Validation binary_cross_entropy = 28.660784
Epoch 16
Validation binary_cross_entropy = 16.150606
Epoch 17
Validation binary_cross_entropy = 28.980709
Epoch 18
Validation binary_cross_entropy = 15.467763
Epoch 19
Loss = 2.1044e+01, PNorm = 83.8271, GNorm = 472.3574, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 8.325538
Epoch 20
Validation binary_cross_entropy = 7.715657
Epoch 21
Validation binary_cross_entropy = 7.511312
Epoch 22
Validation binary_cross_entropy = 11.534036
Epoch 23
Validation binary_cross_entropy = 7.429674
Epoch 24
Loss = 2.4266e+01, PNorm = 84.0211, GNorm = 465.3666, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 6.301119
Epoch 25
Validation binary_cross_entropy = 5.884353
Epoch 26
Validation binary_cross_entropy = 4.574436
Epoch 27
Validation binary_cross_entropy = 5.300784
Epoch 28
Validation binary_cross_entropy = 5.745187
Epoch 29
Loss = 1.3688e+01, PNorm = 84.1576, GNorm = 418.1066, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 12.312782
Epoch 30
Validation binary_cross_entropy = 5.064665
Epoch 31
Validation binary_cross_entropy = 4.420034
Epoch 32
Validation binary_cross_entropy = 5.041101
Epoch 33
Validation binary_cross_entropy = 4.857064
Epoch 34
Loss = 1.6553e+01, PNorm = 84.2764, GNorm = 307.1139, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 7.933010
Epoch 35
Validation binary_cross_entropy = 8.980721
Epoch 36
Validation binary_cross_entropy = 4.908699
Epoch 37
Validation binary_cross_entropy = 5.095532
Epoch 38
Validation binary_cross_entropy = 8.461121
Epoch 39
Loss = 1.4212e+01, PNorm = 84.4096, GNorm = 472.8062, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 14.150459
Epoch 40
Validation binary_cross_entropy = 5.061861
Epoch 41
Validation binary_cross_entropy = 4.613766
Epoch 42
Validation binary_cross_entropy = 10.618230
Epoch 43
Validation binary_cross_entropy = 8.861327
Epoch 44
Loss = 6.8511e+00, PNorm = 84.5207, GNorm = 246.5801, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 5.184186
Epoch 45
Validation binary_cross_entropy = 4.410365
Epoch 46
Validation binary_cross_entropy = 4.155406
Epoch 47
Validation binary_cross_entropy = 6.827839
Epoch 48
Validation binary_cross_entropy = 11.385896
Epoch 49
Loss = 1.0580e+01, PNorm = 84.6309, GNorm = 203.1515, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 5.927272
Epoch 50
Validation binary_cross_entropy = 5.829670
Epoch 51
Validation binary_cross_entropy = 6.870915
Epoch 52
Validation binary_cross_entropy = 6.149588
Epoch 53
Validation binary_cross_entropy = 4.476309
Epoch 54
Loss = 9.5647e+00, PNorm = 84.7297, GNorm = 303.5637, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 3.153519
Epoch 55
Validation binary_cross_entropy = 3.938184
Epoch 56
Validation binary_cross_entropy = 17.610727
Epoch 57
Validation binary_cross_entropy = 12.831160
Epoch 58
Validation binary_cross_entropy = 4.521771
Epoch 59
Loss = 7.3677e+00, PNorm = 84.8190, GNorm = 172.8221, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 2.650998
Epoch 60
Validation binary_cross_entropy = 2.986452
Epoch 61
Validation binary_cross_entropy = 4.442081
Epoch 62
Validation binary_cross_entropy = 4.370672
Epoch 63
Validation binary_cross_entropy = 3.702623
Epoch 64
Loss = 3.0794e+00, PNorm = 84.9076, GNorm = 113.6730, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 3.024895
Epoch 65
Validation binary_cross_entropy = 2.976855
Epoch 66
Validation binary_cross_entropy = 5.284506
Epoch 67
Validation binary_cross_entropy = 6.496588
Epoch 68
Validation binary_cross_entropy = 2.927572
Epoch 69
Loss = 4.0060e+00, PNorm = 85.0045, GNorm = 119.0379, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.055898
Epoch 70
Validation binary_cross_entropy = 2.130948
Epoch 71
Validation binary_cross_entropy = 2.635780
Epoch 72
Validation binary_cross_entropy = 2.114890
Epoch 73
Validation binary_cross_entropy = 3.371248
Epoch 74
Loss = 2.9807e+00, PNorm = 85.1466, GNorm = 77.9959, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 4.933940
Epoch 75
Validation binary_cross_entropy = 5.491914
Epoch 76
Validation binary_cross_entropy = 2.610105
Epoch 77
Validation binary_cross_entropy = 1.649863
Epoch 78
Validation binary_cross_entropy = 1.509089
Epoch 79
Loss = 3.5329e+00, PNorm = 85.2594, GNorm = 199.2752, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 2.555411
Epoch 80
Validation binary_cross_entropy = 2.548480
Epoch 81
Validation binary_cross_entropy = 2.516301
Epoch 82
Validation binary_cross_entropy = 1.882380
Epoch 83
Validation binary_cross_entropy = 1.336473
Epoch 84
Loss = 4.7041e+00, PNorm = 85.3873, GNorm = 194.5735, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 1.401054
Epoch 85
Validation binary_cross_entropy = 1.750644
Epoch 86
Validation binary_cross_entropy = 1.590288
Epoch 87
Validation binary_cross_entropy = 1.755025
Epoch 88
Validation binary_cross_entropy = 3.579889
Epoch 89
Loss = 2.3033e+00, PNorm = 85.5581, GNorm = 109.1309, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.630660
Epoch 90
Validation binary_cross_entropy = 1.304874
Epoch 91
Validation binary_cross_entropy = 1.373879
Epoch 92
Validation binary_cross_entropy = 1.134141
Epoch 93
Validation binary_cross_entropy = 0.940644
Epoch 94
Loss = 1.4004e+00, PNorm = 85.7131, GNorm = 67.0529, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.399476
Epoch 95
Validation binary_cross_entropy = 1.601513
Epoch 96
Validation binary_cross_entropy = 1.444102
Epoch 97
Validation binary_cross_entropy = 0.801374
Epoch 98
Validation binary_cross_entropy = 0.839901
Epoch 99
Loss = 2.0903e+00, PNorm = 85.8328, GNorm = 114.1876, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.760632
Epoch 100
Validation binary_cross_entropy = 0.781886
Epoch 101
Validation binary_cross_entropy = 0.829406
Epoch 102
Validation binary_cross_entropy = 0.703515
Epoch 103
Validation binary_cross_entropy = 0.691156
Epoch 104
Loss = 2.3154e+00, PNorm = 85.9715, GNorm = 116.1718, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.689547
Epoch 105
Validation binary_cross_entropy = 0.714811
Epoch 106
Validation binary_cross_entropy = 0.725829
Epoch 107
Validation binary_cross_entropy = 0.811852
Epoch 108
Validation binary_cross_entropy = 0.697639
Epoch 109
Loss = 1.3154e+00, PNorm = 86.1241, GNorm = 83.8210, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.781253
Epoch 110
Validation binary_cross_entropy = 1.933323
Epoch 111
Validation binary_cross_entropy = 0.817313
Epoch 112
Validation binary_cross_entropy = 0.674670
Epoch 113
Validation binary_cross_entropy = 0.684243
Epoch 114
Loss = 3.2165e+00, PNorm = 86.2595, GNorm = 137.1320, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.597028
Epoch 115
Validation binary_cross_entropy = 0.629777
Epoch 116
Validation binary_cross_entropy = 1.180614
Epoch 117
Validation binary_cross_entropy = 1.478581
Epoch 118
Validation binary_cross_entropy = 1.130739
Epoch 119
Loss = 1.9097e+00, PNorm = 86.3939, GNorm = 66.7703, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.663817
Epoch 120
Validation binary_cross_entropy = 0.596436
Epoch 121
Validation binary_cross_entropy = 0.650551
Epoch 122
Validation binary_cross_entropy = 0.602703
Epoch 123
Validation binary_cross_entropy = 0.613454
Epoch 124
Loss = 1.1850e+00, PNorm = 86.5421, GNorm = 84.2258, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.864629
Epoch 125
Validation binary_cross_entropy = 0.943528
Epoch 126
Validation binary_cross_entropy = 0.687102
Epoch 127
Validation binary_cross_entropy = 0.579807
Epoch 128
Validation binary_cross_entropy = 0.590305
Epoch 129
Loss = 8.7737e-01, PNorm = 86.6321, GNorm = 97.8161, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.569325
Epoch 130
Validation binary_cross_entropy = 0.534282
Epoch 131
Validation binary_cross_entropy = 0.585015
Epoch 132
Validation binary_cross_entropy = 0.713146
Epoch 133
Validation binary_cross_entropy = 0.638562
Epoch 134
Loss = 1.0270e+00, PNorm = 86.7407, GNorm = 69.6633, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.531910
Epoch 135
Validation binary_cross_entropy = 0.473786
Epoch 136
Validation binary_cross_entropy = 0.485731
Epoch 137
Validation binary_cross_entropy = 0.465854
Epoch 138
Validation binary_cross_entropy = 0.451104
Epoch 139
Loss = 6.0874e-01, PNorm = 86.8223, GNorm = 78.3003, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.450130
Epoch 140
Validation binary_cross_entropy = 0.453548
Epoch 141
Validation binary_cross_entropy = 0.465320
Epoch 142
Validation binary_cross_entropy = 0.474434
Epoch 143
Validation binary_cross_entropy = 0.450995
Epoch 144
Loss = 7.2293e-01, PNorm = 86.9103, GNorm = 54.3741, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.441148
Epoch 145
Validation binary_cross_entropy = 0.435210
Epoch 146
Validation binary_cross_entropy = 0.429789
Epoch 147
Validation binary_cross_entropy = 0.433271
Epoch 148
Validation binary_cross_entropy = 0.426706
Epoch 149
Loss = 6.1355e-01, PNorm = 86.9868, GNorm = 64.6853, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.427747
Epoch 150
Validation binary_cross_entropy = 0.422338
Epoch 151
Validation binary_cross_entropy = 0.423104
Epoch 152
Validation binary_cross_entropy = 0.427230
Epoch 153
Validation binary_cross_entropy = 0.448967
Epoch 154
Loss = 6.9319e-01, PNorm = 87.0449, GNorm = 69.5842, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.511370
Epoch 155
Validation binary_cross_entropy = 0.536195
Epoch 156
Validation binary_cross_entropy = 0.542807
Epoch 157
Validation binary_cross_entropy = 0.526077
Epoch 158
Validation binary_cross_entropy = 0.458634
Epoch 159
Loss = 9.8410e-01, PNorm = 87.1184, GNorm = 66.5841, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.436645
Epoch 160
Validation binary_cross_entropy = 0.447494
Epoch 161
Validation binary_cross_entropy = 0.450251
Epoch 162
Validation binary_cross_entropy = 0.447260
Epoch 163
Validation binary_cross_entropy = 0.462330
Epoch 164
Loss = 7.7650e-01, PNorm = 87.1874, GNorm = 57.5813, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.474959
Epoch 165
Validation binary_cross_entropy = 0.480376
Epoch 166
Validation binary_cross_entropy = 0.470453
Epoch 167
Validation binary_cross_entropy = 0.449937
Epoch 168
Validation binary_cross_entropy = 0.453977
Epoch 169
Loss = 9.1834e-01, PNorm = 87.2837, GNorm = 88.6007, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.453212
Epoch 170
Validation binary_cross_entropy = 0.450995
Epoch 171
Validation binary_cross_entropy = 0.466650
Epoch 172
Validation binary_cross_entropy = 0.493154
Epoch 173
Validation binary_cross_entropy = 0.489291
Epoch 174
Loss = 1.1137e+00, PNorm = 87.3874, GNorm = 54.5647, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.462840
Epoch 175
Validation binary_cross_entropy = 0.442760
Epoch 176
Validation binary_cross_entropy = 0.430194
Epoch 177
Validation binary_cross_entropy = 0.425711
Epoch 178
Validation binary_cross_entropy = 0.427930
Epoch 179
Loss = 4.7447e-01, PNorm = 87.5627, GNorm = 38.1573, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.445995
Epoch 180
Validation binary_cross_entropy = 0.408064
Epoch 181
Validation binary_cross_entropy = 0.426542
Epoch 182
Validation binary_cross_entropy = 0.410921
Epoch 183
Validation binary_cross_entropy = 0.380864
Epoch 184
Loss = 8.0860e-01, PNorm = 87.6780, GNorm = 85.5968, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.383110
Epoch 185
Validation binary_cross_entropy = 0.391291
Epoch 186
Validation binary_cross_entropy = 0.440681
Epoch 187
Validation binary_cross_entropy = 0.477558
Epoch 188
Validation binary_cross_entropy = 0.457303
Epoch 189
Loss = 5.0381e-01, PNorm = 87.8136, GNorm = 46.8068, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.404478
Epoch 190
Validation binary_cross_entropy = 0.384730
Epoch 191
Validation binary_cross_entropy = 0.393021
Epoch 192
Validation binary_cross_entropy = 0.428928
Epoch 193
Validation binary_cross_entropy = 0.424160
Epoch 194
Loss = 3.7460e-01, PNorm = 87.9587, GNorm = 36.1049, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.432966
Epoch 195
Validation binary_cross_entropy = 0.422503
Epoch 196
Validation binary_cross_entropy = 0.394919
Epoch 197
Validation binary_cross_entropy = 0.363239
Epoch 198
Validation binary_cross_entropy = 0.353766
Epoch 199
Loss = 4.9800e-01, PNorm = 88.0836, GNorm = 54.4478, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.350624
Epoch 200
Validation binary_cross_entropy = 0.379294
Epoch 201
Validation binary_cross_entropy = 0.416094
Epoch 202
Validation binary_cross_entropy = 0.402245
Epoch 203
Validation binary_cross_entropy = 0.412285
Epoch 204
Loss = 5.2209e-01, PNorm = 88.1906, GNorm = 35.4683, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.398116
Epoch 205
Validation binary_cross_entropy = 0.414095
Epoch 206
Validation binary_cross_entropy = 0.377293
Epoch 207
Validation binary_cross_entropy = 0.358753
Epoch 208
Validation binary_cross_entropy = 0.356279
Epoch 209
Loss = 5.9287e-01, PNorm = 88.3039, GNorm = 24.8477, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.351355
Epoch 210
Validation binary_cross_entropy = 0.355896
Epoch 211
Validation binary_cross_entropy = 0.359411
Epoch 212
Validation binary_cross_entropy = 0.355284
Epoch 213
Validation binary_cross_entropy = 0.341373
Epoch 214
Loss = 2.1941e-01, PNorm = 88.4065, GNorm = 30.7183, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.334156
Epoch 215
Validation binary_cross_entropy = 0.331647
Epoch 216
Validation binary_cross_entropy = 0.334472
Epoch 217
Validation binary_cross_entropy = 0.346832
Epoch 218
Validation binary_cross_entropy = 0.338933
Epoch 219
Loss = 6.9472e-01, PNorm = 88.5261, GNorm = 23.2839, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.329382
Epoch 220
Validation binary_cross_entropy = 0.328917
Epoch 221
Validation binary_cross_entropy = 0.333068
Epoch 222
Validation binary_cross_entropy = 0.325721
Epoch 223
Validation binary_cross_entropy = 0.489592
Epoch 224
Loss = 2.7752e-01, PNorm = 88.6449, GNorm = 63.6143, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.551990
Epoch 225
Validation binary_cross_entropy = 0.373549
Epoch 226
Validation binary_cross_entropy = 0.305024
Epoch 227
Validation binary_cross_entropy = 0.306025
Epoch 228
Validation binary_cross_entropy = 0.314659
Epoch 229
Loss = 6.0461e-01, PNorm = 88.7609, GNorm = 30.7216, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.305157
Epoch 230
Validation binary_cross_entropy = 0.307370
Epoch 231
Validation binary_cross_entropy = 0.341916
Epoch 232
Validation binary_cross_entropy = 0.370227
Epoch 233
Validation binary_cross_entropy = 0.369316
Epoch 234
Loss = 2.7885e-01, PNorm = 88.9257, GNorm = 37.4680, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.350391
Epoch 235
Validation binary_cross_entropy = 0.317019
Epoch 236
Validation binary_cross_entropy = 0.307364
Epoch 237
Validation binary_cross_entropy = 0.316269
Epoch 238
Validation binary_cross_entropy = 0.314924
Epoch 239
Loss = 4.5893e-01, PNorm = 89.0304, GNorm = 33.6235, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.305971
Epoch 240
Validation binary_cross_entropy = 0.313679
Epoch 241
Validation binary_cross_entropy = 0.327067
Epoch 242
Validation binary_cross_entropy = 0.331971
Epoch 243
Validation binary_cross_entropy = 0.331312
Epoch 244
Loss = 3.3576e-01, PNorm = 89.1202, GNorm = 29.6410, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.316621
Epoch 245
Validation binary_cross_entropy = 0.313326
Epoch 246
Validation binary_cross_entropy = 0.311492
Epoch 247
Validation binary_cross_entropy = 0.315834
Epoch 248
Validation binary_cross_entropy = 0.318854
Epoch 249
Loss = 2.4435e-01, PNorm = 89.1848, GNorm = 18.2697, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.327737
Epoch 250
Validation binary_cross_entropy = 0.345818
Epoch 251
Validation binary_cross_entropy = 0.368117
Epoch 252
Validation binary_cross_entropy = 0.374054
Epoch 253
Validation binary_cross_entropy = 0.352314
Epoch 254
Loss = 2.2065e-01, PNorm = 89.2817, GNorm = 13.2514, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.332214
Epoch 255
Validation binary_cross_entropy = 0.330078
Epoch 256
Validation binary_cross_entropy = 0.332736
Epoch 257
Validation binary_cross_entropy = 0.350431
Epoch 258
Validation binary_cross_entropy = 0.375979
Epoch 259
Loss = 4.4584e-01, PNorm = 89.3889, GNorm = 24.4872, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.363681
Epoch 260
Validation binary_cross_entropy = 0.353980
Epoch 261
Validation binary_cross_entropy = 0.350628
Epoch 262
Validation binary_cross_entropy = 0.347419
Epoch 263
Validation binary_cross_entropy = 0.350156
Epoch 264
Loss = 2.6520e-01, PNorm = 89.4614, GNorm = 23.1155, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.352894
Epoch 265
Validation binary_cross_entropy = 0.354109
Epoch 266
Validation binary_cross_entropy = 0.351941
Epoch 267
Validation binary_cross_entropy = 0.361136
Epoch 268
Validation binary_cross_entropy = 0.366441
Epoch 269
Loss = 2.4067e-01, PNorm = 89.5505, GNorm = 29.2051, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.360601
Epoch 270
Validation binary_cross_entropy = 0.353966
Epoch 271
Validation binary_cross_entropy = 0.348387
Epoch 272
Validation binary_cross_entropy = 0.340690
Epoch 273
Validation binary_cross_entropy = 0.342939
Epoch 274
Loss = 1.1608e-01, PNorm = 89.6189, GNorm = 8.2515, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.350268
Epoch 275
Validation binary_cross_entropy = 0.355458
Epoch 276
Validation binary_cross_entropy = 0.356676
Epoch 277
Validation binary_cross_entropy = 0.364518
Epoch 278
Validation binary_cross_entropy = 0.388073
Epoch 279
Loss = 4.6534e-01, PNorm = 89.7092, GNorm = 33.7491, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.413855
Epoch 280
Validation binary_cross_entropy = 0.414461
Epoch 281
Validation binary_cross_entropy = 0.394352
Epoch 282
Validation binary_cross_entropy = 0.368184
Epoch 283
Validation binary_cross_entropy = 0.350186
Epoch 284
Loss = 1.9674e-01, PNorm = 89.7922, GNorm = 8.4280, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.344368
Epoch 285
Validation binary_cross_entropy = 0.342854
Epoch 286
Validation binary_cross_entropy = 0.344009
Epoch 287
Validation binary_cross_entropy = 0.350203
Epoch 288
Validation binary_cross_entropy = 0.368348
Epoch 289
Loss = 1.5286e-01, PNorm = 89.9187, GNorm = 6.8245, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.385089
Epoch 290
Validation binary_cross_entropy = 0.386250
Epoch 291
Validation binary_cross_entropy = 0.382176
Epoch 292
Validation binary_cross_entropy = 0.375893
Epoch 293
Validation binary_cross_entropy = 0.376457
Epoch 294
Loss = 5.0250e-01, PNorm = 90.0527, GNorm = 55.4746, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.368699
Epoch 295
Validation binary_cross_entropy = 0.400451
Epoch 296
Validation binary_cross_entropy = 0.456183
Epoch 297
Validation binary_cross_entropy = 0.413230
Epoch 298
Validation binary_cross_entropy = 0.362241
Epoch 299
Loss = 2.6265e-01, PNorm = 90.1668, GNorm = 37.7953, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.351281
Model 0 best validation binary_cross_entropy = 0.305024 on epoch 226
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.194432
Ensemble test binary_cross_entropy = 0.194432
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1600, bias=False)
        (W_h): Linear(in_features=1600, out_features=1600, bias=False)
        (W_o): Linear(in_features=1733, out_features=1600, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=1900, out_features=1600, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1600, out_features=1600, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.35000000000000003, inplace=False)
    (7): Linear(in_features=1600, out_features=1, bias=True)
  )
)
Number of parameters = 11,174,401
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 47.779530
Epoch 1
Validation binary_cross_entropy = 4.155210
Epoch 2
Validation binary_cross_entropy = 69.169006
Epoch 3
Validation binary_cross_entropy = 15.244905
Epoch 4
Loss = 3.0908e+01, PNorm = 82.9672, GNorm = 592.2845, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 12.479959
Epoch 5
Validation binary_cross_entropy = 52.136013
Epoch 6
Validation binary_cross_entropy = 5.738629
Epoch 7
Validation binary_cross_entropy = 13.245847
Epoch 8
Validation binary_cross_entropy = 9.299591
Epoch 9
Loss = 3.1398e+01, PNorm = 83.2685, GNorm = 813.0681, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 10.382499
Epoch 10
Validation binary_cross_entropy = 17.295446
Epoch 11
Validation binary_cross_entropy = 6.570008
Epoch 12
Validation binary_cross_entropy = 47.552757
Epoch 13
Validation binary_cross_entropy = 13.169567
Epoch 14
Loss = 3.0033e+01, PNorm = 83.6081, GNorm = 698.1564, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 6.283323
Epoch 15
Validation binary_cross_entropy = 33.306538
Epoch 16
Validation binary_cross_entropy = 29.671965
Epoch 17
Validation binary_cross_entropy = 5.637748
Epoch 18
Validation binary_cross_entropy = 11.614808
Epoch 19
Loss = 2.3042e+01, PNorm = 83.8902, GNorm = 852.6582, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 47.905499
Epoch 20
Validation binary_cross_entropy = 7.011623
Epoch 21
Validation binary_cross_entropy = 6.674687
Epoch 22
Validation binary_cross_entropy = 33.501339
Epoch 23
Validation binary_cross_entropy = 42.539177
Epoch 24
Loss = 1.3127e+01, PNorm = 84.0731, GNorm = 212.3424, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 10.597239
Epoch 25
Validation binary_cross_entropy = 8.188497
Epoch 26
Validation binary_cross_entropy = 10.790643
Epoch 27
Validation binary_cross_entropy = 10.175017
Epoch 28
Validation binary_cross_entropy = 16.731159
Epoch 29
Loss = 7.7829e+00, PNorm = 84.1850, GNorm = 295.1528, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 24.509079
Epoch 30
Validation binary_cross_entropy = 7.874010
Epoch 31
Validation binary_cross_entropy = 5.832706
Epoch 32
Validation binary_cross_entropy = 7.903533
Epoch 33
Validation binary_cross_entropy = 10.827582
Epoch 34
Loss = 1.1342e+01, PNorm = 84.2639, GNorm = 467.3612, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 8.860486
Epoch 35
Validation binary_cross_entropy = 5.622736
Epoch 36
Validation binary_cross_entropy = 6.339694
Epoch 37
Validation binary_cross_entropy = 22.843206
Epoch 38
Validation binary_cross_entropy = 6.435801
Epoch 39
Loss = 1.2853e+01, PNorm = 84.3254, GNorm = 308.2395, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 5.118277
Epoch 40
Validation binary_cross_entropy = 4.835630
Epoch 41
Validation binary_cross_entropy = 14.216537
Epoch 42
Validation binary_cross_entropy = 13.034728
Epoch 43
Validation binary_cross_entropy = 5.738956
Epoch 44
Loss = 9.1605e+00, PNorm = 84.3865, GNorm = 288.0389, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 4.133779
Epoch 45
Validation binary_cross_entropy = 4.105047
Epoch 46
Validation binary_cross_entropy = 17.907667
Epoch 47
Validation binary_cross_entropy = 14.458001
Epoch 48
Validation binary_cross_entropy = 3.745667
Epoch 49
Loss = 8.1024e+00, PNorm = 84.4515, GNorm = 211.3030, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 3.722748
Epoch 50
Validation binary_cross_entropy = 4.030944
Epoch 51
Validation binary_cross_entropy = 5.909028
Epoch 52
Validation binary_cross_entropy = 11.997333
Epoch 53
Validation binary_cross_entropy = 13.272811
Epoch 54
Loss = 1.0633e+01, PNorm = 84.5252, GNorm = 321.1023, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 3.806582
Epoch 55
Validation binary_cross_entropy = 3.144754
Epoch 56
Validation binary_cross_entropy = 3.107319
Epoch 57
Validation binary_cross_entropy = 6.284720
Epoch 58
Validation binary_cross_entropy = 10.207974
Epoch 59
Loss = 6.0055e+00, PNorm = 84.6021, GNorm = 290.3911, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 3.484590
Epoch 60
Validation binary_cross_entropy = 3.234117
Epoch 61
Validation binary_cross_entropy = 2.539685
Epoch 62
Validation binary_cross_entropy = 3.047108
Epoch 63
Validation binary_cross_entropy = 2.570588
Epoch 64
Loss = 4.6854e+00, PNorm = 84.6826, GNorm = 149.7450, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 3.957509
Epoch 65
Validation binary_cross_entropy = 3.478778
Epoch 66
Validation binary_cross_entropy = 2.451099
Epoch 67
Validation binary_cross_entropy = 1.917418
Epoch 68
Validation binary_cross_entropy = 1.872276
Epoch 69
Loss = 2.9312e+00, PNorm = 84.7541, GNorm = 89.2446, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 2.030989
Epoch 70
Validation binary_cross_entropy = 4.554156
Epoch 71
Validation binary_cross_entropy = 5.057094
Epoch 72
Validation binary_cross_entropy = 3.129266
Epoch 73
Validation binary_cross_entropy = 2.043193
Epoch 74
Loss = 2.8675e+00, PNorm = 84.8399, GNorm = 152.6536, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 2.092969
Epoch 75
Validation binary_cross_entropy = 2.625432
Epoch 76
Validation binary_cross_entropy = 3.125352
Epoch 77
Validation binary_cross_entropy = 3.083823
Epoch 78
Validation binary_cross_entropy = 3.062429
Epoch 79
Loss = 3.1990e+00, PNorm = 84.9506, GNorm = 132.1273, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 3.044752
Epoch 80
Validation binary_cross_entropy = 3.013565
Epoch 81
Validation binary_cross_entropy = 3.013576
Epoch 82
Validation binary_cross_entropy = 3.017519
Epoch 83
Validation binary_cross_entropy = 3.025645
Epoch 84
Loss = 4.1884e+00, PNorm = 85.0482, GNorm = 112.7041, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 3.009189
Epoch 85
Validation binary_cross_entropy = 3.008991
Epoch 86
Validation binary_cross_entropy = 2.975894
Epoch 87
Validation binary_cross_entropy = 1.750741
Epoch 88
Validation binary_cross_entropy = 1.708500
Epoch 89
Loss = 2.6672e+00, PNorm = 85.1869, GNorm = 64.1783, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.688489
Epoch 90
Validation binary_cross_entropy = 1.569526
Epoch 91
Validation binary_cross_entropy = 0.962361
Epoch 92
Validation binary_cross_entropy = 0.924600
Epoch 93
Validation binary_cross_entropy = 0.895766
Epoch 94
Loss = 1.5327e+00, PNorm = 85.3325, GNorm = 105.7340, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.891241
Epoch 95
Validation binary_cross_entropy = 0.947592
Epoch 96
Validation binary_cross_entropy = 1.789183
Epoch 97
Validation binary_cross_entropy = 3.018553
Epoch 98
Validation binary_cross_entropy = 1.596593
Epoch 99
Loss = 2.2307e+00, PNorm = 85.4361, GNorm = 185.3232, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.108262
Epoch 100
Validation binary_cross_entropy = 1.170071
Epoch 101
Validation binary_cross_entropy = 0.971141
Epoch 102
Validation binary_cross_entropy = 1.609923
Epoch 103
Validation binary_cross_entropy = 2.292602
Epoch 104
Loss = 2.7076e+00, PNorm = 85.5563, GNorm = 187.4788, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.582426
Epoch 105
Validation binary_cross_entropy = 1.437777
Epoch 106
Validation binary_cross_entropy = 0.875241
Epoch 107
Validation binary_cross_entropy = 0.859603
Epoch 108
Validation binary_cross_entropy = 0.878825
Epoch 109
Loss = 1.7381e+00, PNorm = 85.6595, GNorm = 97.0387, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 2.064111
Epoch 110
Validation binary_cross_entropy = 2.191100
Epoch 111
Validation binary_cross_entropy = 1.475043
Epoch 112
Validation binary_cross_entropy = 0.798614
Epoch 113
Validation binary_cross_entropy = 0.771553
Epoch 114
Loss = 1.4807e+00, PNorm = 85.7138, GNorm = 38.9152, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.803846
Epoch 115
Validation binary_cross_entropy = 0.887421
Epoch 116
Validation binary_cross_entropy = 1.506570
Epoch 117
Validation binary_cross_entropy = 1.564665
Epoch 118
Validation binary_cross_entropy = 0.879360
Epoch 119
Loss = 9.1791e-01, PNorm = 85.7808, GNorm = 54.1254, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.754685
Epoch 120
Validation binary_cross_entropy = 0.705533
Epoch 121
Validation binary_cross_entropy = 0.702702
Epoch 122
Validation binary_cross_entropy = 0.842947
Epoch 123
Validation binary_cross_entropy = 1.176755
Epoch 124
Loss = 1.3512e+00, PNorm = 85.8640, GNorm = 67.5193, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.450489
Epoch 125
Validation binary_cross_entropy = 1.180570
Epoch 126
Validation binary_cross_entropy = 1.000685
Epoch 127
Validation binary_cross_entropy = 0.869627
Epoch 128
Validation binary_cross_entropy = 0.750740
Epoch 129
Loss = 9.5501e-01, PNorm = 85.9426, GNorm = 61.1310, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.773293
Epoch 130
Validation binary_cross_entropy = 0.737562
Epoch 131
Validation binary_cross_entropy = 0.730382
Epoch 132
Validation binary_cross_entropy = 0.663510
Epoch 133
Validation binary_cross_entropy = 0.635893
Epoch 134
Loss = 9.1592e-01, PNorm = 86.0296, GNorm = 51.6351, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.624092
Epoch 135
Validation binary_cross_entropy = 0.628693
Epoch 136
Validation binary_cross_entropy = 0.666791
Epoch 137
Validation binary_cross_entropy = 0.708765
Epoch 138
Validation binary_cross_entropy = 0.666523
Epoch 139
Loss = 9.9702e-01, PNorm = 86.0996, GNorm = 67.7058, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.644398
Epoch 140
Validation binary_cross_entropy = 0.619459
Epoch 141
Validation binary_cross_entropy = 0.627820
Epoch 142
Validation binary_cross_entropy = 0.717261
Epoch 143
Validation binary_cross_entropy = 0.784959
Epoch 144
Loss = 1.0505e+00, PNorm = 86.2018, GNorm = 88.7590, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.741364
Epoch 145
Validation binary_cross_entropy = 0.668111
Epoch 146
Validation binary_cross_entropy = 0.644205
Epoch 147
Validation binary_cross_entropy = 0.638251
Epoch 148
Validation binary_cross_entropy = 0.602964
Epoch 149
Loss = 1.0264e+00, PNorm = 86.3024, GNorm = 57.9200, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.585372
Epoch 150
Validation binary_cross_entropy = 0.577371
Epoch 151
Validation binary_cross_entropy = 0.552800
Epoch 152
Validation binary_cross_entropy = 0.564304
Epoch 153
Validation binary_cross_entropy = 0.609546
Epoch 154
Loss = 1.5465e+00, PNorm = 86.4509, GNorm = 91.0139, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.532803
Epoch 155
Validation binary_cross_entropy = 0.491973
Epoch 156
Validation binary_cross_entropy = 0.491373
Epoch 157
Validation binary_cross_entropy = 0.518653
Epoch 158
Validation binary_cross_entropy = 0.540854
Epoch 159
Loss = 8.8880e-01, PNorm = 86.5788, GNorm = 66.2066, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.563474
Epoch 160
Validation binary_cross_entropy = 0.515174
Epoch 161
Validation binary_cross_entropy = 0.499341
Epoch 162
Validation binary_cross_entropy = 0.503402
Epoch 163
Validation binary_cross_entropy = 0.518216
Epoch 164
Loss = 8.4460e-01, PNorm = 86.6853, GNorm = 56.5912, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.556593
Epoch 165
Validation binary_cross_entropy = 0.623949
Epoch 166
Validation binary_cross_entropy = 0.595669
Epoch 167
Validation binary_cross_entropy = 0.486896
Epoch 168
Validation binary_cross_entropy = 0.454798
Epoch 169
Loss = 4.9647e-01, PNorm = 86.7812, GNorm = 64.6433, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.456134
Epoch 170
Validation binary_cross_entropy = 0.459137
Epoch 171
Validation binary_cross_entropy = 0.485694
Epoch 172
Validation binary_cross_entropy = 0.545842
Epoch 173
Validation binary_cross_entropy = 0.598922
Epoch 174
Loss = 2.9432e-01, PNorm = 86.8820, GNorm = 42.6523, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.654751
Epoch 175
Validation binary_cross_entropy = 0.564606
Epoch 176
Validation binary_cross_entropy = 0.515328
Epoch 177
Validation binary_cross_entropy = 0.487032
Epoch 178
Validation binary_cross_entropy = 0.480682
Epoch 179
Loss = 1.1743e+00, PNorm = 86.9476, GNorm = 49.1034, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.507892
Epoch 180
Validation binary_cross_entropy = 0.536714
Epoch 181
Validation binary_cross_entropy = 0.590752
Epoch 182
Validation binary_cross_entropy = 0.602050
Epoch 183
Validation binary_cross_entropy = 0.537326
Epoch 184
Loss = 8.3017e-01, PNorm = 87.0429, GNorm = 57.1505, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.476434
Epoch 185
Validation binary_cross_entropy = 0.439445
Epoch 186
Validation binary_cross_entropy = 0.440799
Epoch 187
Validation binary_cross_entropy = 0.447491
Epoch 188
Validation binary_cross_entropy = 0.441658
Epoch 189
Loss = 5.1723e-01, PNorm = 87.1358, GNorm = 28.8490, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.431290
Epoch 190
Validation binary_cross_entropy = 0.440063
Epoch 191
Validation binary_cross_entropy = 0.464869
Epoch 192
Validation binary_cross_entropy = 0.473091
Epoch 193
Validation binary_cross_entropy = 0.487809
Epoch 194
Loss = 3.3887e-01, PNorm = 87.2282, GNorm = 25.9565, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.503069
Epoch 195
Validation binary_cross_entropy = 0.528645
Epoch 196
Validation binary_cross_entropy = 0.486687
Epoch 197
Validation binary_cross_entropy = 0.457089
Epoch 198
Validation binary_cross_entropy = 0.465028
Epoch 199
Loss = 9.2229e-01, PNorm = 87.3166, GNorm = 72.2960, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.475859
Epoch 200
Validation binary_cross_entropy = 0.569393
Epoch 201
Validation binary_cross_entropy = 0.660037
Epoch 202
Validation binary_cross_entropy = 0.580005
Epoch 203
Validation binary_cross_entropy = 0.447250
Epoch 204
Loss = 5.9047e-01, PNorm = 87.4379, GNorm = 41.6570, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.393796
Epoch 205
Validation binary_cross_entropy = 0.383258
Epoch 206
Validation binary_cross_entropy = 0.382164
Epoch 207
Validation binary_cross_entropy = 0.394843
Epoch 208
Validation binary_cross_entropy = 0.397973
Epoch 209
Loss = 6.4751e-01, PNorm = 87.5704, GNorm = 41.9071, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.401937
Epoch 210
Validation binary_cross_entropy = 0.415941
Epoch 211
Validation binary_cross_entropy = 0.421101
Epoch 212
Validation binary_cross_entropy = 0.427682
Epoch 213
Validation binary_cross_entropy = 0.433840
Epoch 214
Loss = 7.2229e-01, PNorm = 87.7022, GNorm = 56.5323, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.457393
Epoch 215
Validation binary_cross_entropy = 0.476590
Epoch 216
Validation binary_cross_entropy = 0.441899
Epoch 217
Validation binary_cross_entropy = 0.433898
Epoch 218
Validation binary_cross_entropy = 0.502772
Epoch 219
Loss = 4.9872e-01, PNorm = 87.8057, GNorm = 23.9529, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.505183
Epoch 220
Validation binary_cross_entropy = 0.515474
Epoch 221
Validation binary_cross_entropy = 0.673235
Epoch 222
Validation binary_cross_entropy = 0.884126
Epoch 223
Validation binary_cross_entropy = 0.798726
Epoch 224
Loss = 4.7179e-01, PNorm = 87.9257, GNorm = 21.3937, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.620308
Epoch 225
Validation binary_cross_entropy = 0.500590
Epoch 226
Validation binary_cross_entropy = 0.443208
Epoch 227
Validation binary_cross_entropy = 0.421903
Epoch 228
Validation binary_cross_entropy = 0.436979
Epoch 229
Loss = 3.5667e-01, PNorm = 88.0229, GNorm = 24.1688, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.440240
Epoch 230
Validation binary_cross_entropy = 0.434676
Epoch 231
Validation binary_cross_entropy = 0.434854
Epoch 232
Validation binary_cross_entropy = 0.413819
Epoch 233
Validation binary_cross_entropy = 0.397369
Epoch 234
Loss = 3.8442e-01, PNorm = 88.1537, GNorm = 11.2478, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.387122
Epoch 235
Validation binary_cross_entropy = 0.406788
Epoch 236
Validation binary_cross_entropy = 0.475991
Epoch 237
Validation binary_cross_entropy = 0.515299
Epoch 238
Validation binary_cross_entropy = 0.495671
Epoch 239
Loss = 8.0274e-01, PNorm = 88.3172, GNorm = 47.4007, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.437423
Epoch 240
Validation binary_cross_entropy = 0.414318
Epoch 241
Validation binary_cross_entropy = 0.404633
Epoch 242
Validation binary_cross_entropy = 0.399527
Epoch 243
Validation binary_cross_entropy = 0.397609
Epoch 244
Loss = 2.9848e-01, PNorm = 88.4440, GNorm = 23.1271, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.400255
Epoch 245
Validation binary_cross_entropy = 0.416206
Epoch 246
Validation binary_cross_entropy = 0.467800
Epoch 247
Validation binary_cross_entropy = 0.466858
Epoch 248
Validation binary_cross_entropy = 0.408801
Epoch 249
Loss = 1.9075e-01, PNorm = 88.5769, GNorm = 24.8927, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.380238
Epoch 250
Validation binary_cross_entropy = 0.380035
Epoch 251
Validation binary_cross_entropy = 0.381895
Epoch 252
Validation binary_cross_entropy = 0.405303
Epoch 253
Validation binary_cross_entropy = 0.458566
Epoch 254
Loss = 4.1102e-01, PNorm = 88.7247, GNorm = 25.3135, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.502867
Epoch 255
Validation binary_cross_entropy = 0.517387
Epoch 256
Validation binary_cross_entropy = 0.506510
Epoch 257
Validation binary_cross_entropy = 0.443130
Epoch 258
Validation binary_cross_entropy = 0.398543
Epoch 259
Loss = 2.8464e-01, PNorm = 88.8652, GNorm = 27.3676, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.372273
Epoch 260
Validation binary_cross_entropy = 0.367419
Epoch 261
Validation binary_cross_entropy = 0.366393
Epoch 262
Validation binary_cross_entropy = 0.370879
Epoch 263
Validation binary_cross_entropy = 0.405904
Epoch 264
Loss = 4.4453e-01, PNorm = 89.0102, GNorm = 32.7572, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.436773
Epoch 265
Validation binary_cross_entropy = 0.429732
Epoch 266
Validation binary_cross_entropy = 0.399980
Epoch 267
Validation binary_cross_entropy = 0.377013
Epoch 268
Validation binary_cross_entropy = 0.364158
Epoch 269
Loss = 2.3704e-01, PNorm = 89.0955, GNorm = 20.9586, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.361027
Epoch 270
Validation binary_cross_entropy = 0.360845
Epoch 271
Validation binary_cross_entropy = 0.360808
Epoch 272
Validation binary_cross_entropy = 0.364371
Epoch 273
Validation binary_cross_entropy = 0.372518
Epoch 274
Loss = 6.2144e-01, PNorm = 89.1945, GNorm = 39.3683, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.382826
Epoch 275
Validation binary_cross_entropy = 0.392979
Epoch 276
Validation binary_cross_entropy = 0.390683
Epoch 277
Validation binary_cross_entropy = 0.387104
Epoch 278
Validation binary_cross_entropy = 0.370408
Epoch 279
Loss = 2.6444e-01, PNorm = 89.3103, GNorm = 13.5665, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.362894
Epoch 280
Validation binary_cross_entropy = 0.361846
Epoch 281
Validation binary_cross_entropy = 0.373521
Epoch 282
Validation binary_cross_entropy = 0.395105
Epoch 283
Validation binary_cross_entropy = 0.408461
Epoch 284
Loss = 4.7591e-01, PNorm = 89.4154, GNorm = 30.9805, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.389639
Epoch 285
Validation binary_cross_entropy = 0.366234
Epoch 286
Validation binary_cross_entropy = 0.351022
Epoch 287
Validation binary_cross_entropy = 0.344770
Epoch 288
Validation binary_cross_entropy = 0.346033
Epoch 289
Loss = 1.9236e-01, PNorm = 89.5139, GNorm = 14.1226, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.344599
Epoch 290
Validation binary_cross_entropy = 0.351038
Epoch 291
Validation binary_cross_entropy = 0.365884
Epoch 292
Validation binary_cross_entropy = 0.377147
Epoch 293
Validation binary_cross_entropy = 0.382853
Epoch 294
Loss = 5.4673e-01, PNorm = 89.6283, GNorm = 47.4773, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.364381
Epoch 295
Validation binary_cross_entropy = 0.352718
Epoch 296
Validation binary_cross_entropy = 0.350791
Epoch 297
Validation binary_cross_entropy = 0.354219
Epoch 298
Validation binary_cross_entropy = 0.362339
Epoch 299
Loss = 3.1848e-01, PNorm = 89.7421, GNorm = 20.8848, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.367867
Model 0 best validation binary_cross_entropy = 0.344599 on epoch 289
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.156168
Ensemble test binary_cross_entropy = 0.156168
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.180758
	Seed 1 ==> test binary_cross_entropy = 0.182712
	Seed 2 ==> test binary_cross_entropy = 0.179068
	Seed 3 ==> test binary_cross_entropy = 0.168620
	Seed 4 ==> test binary_cross_entropy = 0.156751
	Seed 5 ==> test binary_cross_entropy = 0.217674
	Seed 6 ==> test binary_cross_entropy = 0.170909
	Seed 7 ==> test binary_cross_entropy = 0.162515
	Seed 8 ==> test binary_cross_entropy = 0.194432
	Seed 9 ==> test binary_cross_entropy = 0.156168
Overall test binary_cross_entropy = 0.176961 +/- 0.017806
Elapsed time = 0:05:35
